← Back to papers
2016 ICML ICML 2016

Solving Ridge Regression using Sketched Preconditioned SVRG

Abstract

We develop a novel preconditioning method for ridge regression, based on recent linear sketching methods. By equipping Stochastic Variance Reduced Gradient (SVRG) with this preconditioning process, we obtain a significant speed-up relative to fast stochastic methods such as SVRG, SDCA and SAG.

🌉 Interdisciplinary Bridge — Machine Learning and Mathematics & Optimization
🧭 Keyword Pioneer — numerical optimization
🐝 Cross-Pollinator — Artificial Intelligence, Machine Learning, Mathematics & Optimization, Natural Language Processing
📈 Trend Setter — Numerical Analysis

Authors

Alon Gonen , Francesco Orabona , Shai Shalev-shwartz

Topics

Machine Learning > Core Methods > Regression Machine Learning > Optimization & Theory > Neural Network Optimization Machine Learning > Optimization & Theory > Optimization Mathematics & Optimization > Optimization > Continuous Optimization Mathematics & Optimization > Optimization > Numerical Analysis

Keywords

stochastic gradient descent ridge regression numerical optimization stochastic variance reduced gradient linear sketching
Download PDF

Related papers

Associative Long Short-Term Memory 2016
Recycling Randomness with Structure for Sublinear time Kernel Expansions 2016
Stochastically Transitive Models for Pairwise Comparisons: Statistical and Computational Issues 2016
Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization 2016
Hawkes Processes with Stochastic Excitations 2016