2009
JMLR
JMLR 2009
SGD-QN: Careful Quasi-Newton Stochastic Gradient Descent
Abstract
The SGD-QN algorithm is a stochastic gradient descent algorithm that makes careful use of second-order information and splits the parameter update into independently scheduled components. Thanks to this design, SGD-QN iterates nearly as fast as a first-order stochastic gradient descent but requires less iterations to achieve the same accuracy. This algorithm won the "Wild Track" of the first PASCAL Large Scale Learning Challenge (Sonnenburg et al., 2008). [abs] [ pdf ][ bib ] © JMLR 2009. (edit, beta)
🌉
Interdisciplinary Bridge
— Machine Learning and Mathematics & Optimization
📈
Trend Setter
— Neural Network Optimization
🧭
Keyword Pioneer
— second-order information
🐣
Hot Topic Early Bird
— stochastic gradient descent
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning