2023
NIPS
NeurIPS 2023
Improved Convergence in High Probability of Clipped Gradient Methods with Heavy Tailed Noise
Abstract
In this work, we study the convergence in high probability of clipped gradient methods when the noise distribution has heavy tails, i.e., with bounded $p$th moments, for some $1
🌉
Interdisciplinary Bridge
— Deep Learning and Machine Learning and Mathematics & Optimization
🧭
Keyword Pioneer
— high probability convergence
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
Authors
Topics
Machine Learning > Optimization & Theory > Neural Network Optimization
Machine Learning > Optimization & Theory > Optimization
Mathematics & Optimization > Optimization > Stochastic Methods
Machine Learning > Optimization & Theory > Stochastic Methods
Deep Learning > Optimization & Theory > Neural Network Optimization