← Back to papers
2017 JMLR JMLR 2017

Accelerating Stochastic Composition Optimization

Abstract

We consider the stochastic nested composition optimization problem where the objective is a composition of two expected- value functions. We propose a new stochastic first-order method, namely the accelerated stochastic compositional proximal gradient (ASC-PG) method. This algorithm updates the solution based on noisy gradient queries using a two-timescale iteration. The ASC-PG is the first proximal gradient method for the stochastic composition problem that can deal with nonsmooth regularization penalty. We show that the ASC-PG exhibits faster convergence than the best known algorithms, and that it achieves the optimal sample-error complexity in several important special cases. We demonstrate the application of ASC-PG to reinforcement learning and conduct numerical experiments. [abs] [ pdf ][ bib ] © JMLR 2017. (edit, beta)

🌉 Interdisciplinary Bridge — Machine Learning and Reinforcement Learning
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Mengdi Wang , Ji Liu , Ethan X. Fang

Topics

Machine Learning > Optimization & Theory > Neural Network Optimization Machine Learning > Optimization & Theory > Optimization Reinforcement Learning > Methods > Deep RL

Keywords

stochastic optimization reinforcement learning proximal gradient method compositional optimization sample-error complexity
Download PDF

Related papers

On Computationally Tractable Selection of Experiments in Measurement-Constrained Regression Models 2017
Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization 2017
Automatic Differentiation Variational Inference 2017
Empirical Evaluation of Resampling Procedures for Optimising SVM Hyperparameters 2017
Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning 2017