2014
NIPS
NeurIPS 2014
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
Abstract
In this work we introduce a new fast incremental gradient method SAGA, in the spirit of SAG, SDCA, MISO and SVRG. SAGA improves on the theory behind SAG and SVRG, with better theoretical convergence rates, and support for composite objectives where a proximal operator is used on the regulariser. Unlike SDCA, SAGA supports non-strongly convex problems directly, and is adaptive to any inherent strong convexity of the problem. We give experimental results showing the effectiveness of our method.
🌉
Interdisciplinary Bridge
— Deep Learning and Machine Learning and Mathematics & Optimization
🧭
Keyword Pioneer
— incremental gradient
🐣
Hot Topic Early Bird
— stochastic gradient
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
Authors
Topics
Machine Learning > Optimization & Theory > Neural Network Optimization
Machine Learning > Optimization & Theory > Optimization
Mathematics & Optimization > Optimization > Continuous Optimization
Mathematics & Optimization > Optimization > Optimization
Machine Learning > Core Methods > Optimization
Deep Learning > Optimization & Theory > Optimization