← Back to papers
2022 AAAI AAAI 2022

A Stochastic Momentum Accelerated Quasi-Newton Method for Neural Networks (Student Abstract)

Abstract

Abstract Incorporating curvature information in stochastic methods has been a challenging task. This paper proposes a momentum accelerated BFGS quasi-Newton method in both its full and limited memory forms, for solving stochastic large scale non-convex optimization problems in neural networks (NN).

🌉 Interdisciplinary Bridge — Deep Learning and Machine Learning and Mathematics & Optimization
🧭 Keyword Pioneer — bfgs algorithm
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

S. Indrapriyadarsini , Shahrzad Mahboubi , Hiroshi Ninomiya , Takeshi Kamio , Hideki Asai

Topics

Machine Learning > Optimization & Theory > Neural Network Optimization Machine Learning > Optimization & Theory > Optimization Deep Learning > Techniques > Model Architecture Mathematics & Optimization > Optimization > Optimization Deep Learning > Optimization & Theory > Neural Network Optimization Deep Learning > Optimization & Theory > Optimization Machine Learning > Learning Types > Optimization

Keywords

stochastic optimization neural network training quasi-newton method momentum method momentum acceleration neural network bfgs algorithm limited memory optimization
Download PDF

Related papers

Dynamic Spatial Propagation Network for Depth Completion 2022
FedFR: Joint Optimization Federated Framework for Generic and Personalized Face Recognition 2022
Memory-Guided Semantic Learning Network for Temporal Sentence Grounding 2022
AnchorFace: Boosting TAR@FAR for Practical Face Recognition 2022
Parallel and High-Fidelity Text-to-Lip Generation 2022