← Optimization & Theory
Deep Learning › Optimization & Theory ›

Neural Network Optimization

902 directly classified papers

Papers per year

Papers

What Can ResNet Learn Efficiently, Going Beyond Kernels? NIPS 2019
Efficient Forward Architecture Search NIPS 2019
Accelerating Rescaled Gradient Descent: Fast Optimization of Smooth Functions NIPS 2019
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel NIPS 2019
The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks NIPS 2019
Fine-grained Optimization of Deep Neural Networks NIPS 2019
Online Normalization for Training Neural Networks NIPS 2019
Initialization of ReLUs for Dynamical Isometry NIPS 2019
Surfing: Iterative Optimization Over Incrementally Trained Deep Networks NIPS 2019
Global Sparse Momentum SGD for Pruning Very Deep Neural Networks NIPS 2019
Stagewise Training Accelerates Convergence of Testing Error Over SGD NIPS 2019
ChamNet: Towards Efficient Network Design Through Platform-Aware Model Adaptation CVPR 2019
A Sufficient Condition for Convergences of Adam and RMSProp CVPR 2019
IRLAS: Inverse Reinforcement Learning for Architecture Search CVPR 2019
Dynamic Recursive Neural Network CVPR 2019
Searching for a Robust Neural Architecture in Four GPU Hours CVPR 2019
How to Start Training: The Effect of Initialization and Architecture NIPS 2018
Implicit Bias of Gradient Descent on Linear Convolutional Networks NIPS 2018
Hessian-based Analysis of Large Batch Training and Robustness to Adversaries NIPS 2018
Training Deep Neural Networks with 8-bit Floating Point Numbers NIPS 2018
Backpropagation with Callbacks: Foundations for Efficient and Expressive Differentiable Programming NIPS 2018
Fast Approximate Natural Gradient Descent in a Kronecker Factored Eigenbasis NIPS 2018
The Physical Systems Behind Optimization Algorithms NIPS 2018
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data NIPS 2018
Evolutionary Stochastic Gradient Descent for Optimization of Deep Neural Networks NIPS 2018