← Back to papers
2009 NIPS NeurIPS 2009

Slow Learners are Fast

Abstract

Online learning algorithms have impressive convergence properties when it comes to risk minimization and convex games on very large problems. However, they are inherently sequential in their design which prevents them from taking advantage of modern multi-core architectures. In this paper we prove that online learning with delayed updates converges well, thereby facilitating parallel online learning.

🌉 Interdisciplinary Bridge — Machine Learning and Mathematics & Optimization
📈 Trend Setter — Distributed Learning
🧭 Keyword Pioneer — parallel learning
🐣 Hot Topic Early Bird — online learning
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Martin Zinkevich , John Langford , Alex J. Smola

Topics

Machine Learning > Optimization & Theory > Distributed Learning Machine Learning > Optimization & Theory > Learning Theory Machine Learning > Optimization & Theory > Optimization Mathematics & Optimization > Optimization > Online Algorithms Machine Learning > Learning Types > Online Learning Machine Learning > Optimization & Theory > Online Algorithms Machine Learning > Optimization & Theory > Stochastic Methods Deep Learning > Optimization & Theory > Efficient Computing

Keywords

stochastic optimization online learning convex optimization risk minimization parallel learning delayed update parallel online learning convex game
Download PDF

Related papers

Solving Stochastic Games 2009
Bilinear classifiers for visual recognition 2009
Zero-shot Learning with Semantic Output Codes 2009
Matrix Completion from Power-Law Distributed Samples 2009
Heavy-Tailed Symmetric Stochastic Neighbor Embedding 2009