Samuel L Smith

6 papers · 2021–2024 · 2 conferences · across top CS/AI conferences

Achievements

🌉 Interdisciplinary Bridge 🧭 Keyword Pioneer 🌍 Conference Polyglot (2) 🐝 Cross-Pollinator (13) 🐣 Hot Topic Early Bird

Conferences

ICLR (3) ICML (3)

Top co-authors

Soham De (5) Caglar Gulcehre (2) Antonio Orvieto (2) Andrew Brock (2) Razvan Pascanu (2) Karen Simonyan (1) David Barrett (1) Yee Whye Teh (1) Aleksandar Botev (1) Anushan Fernando (1)

Research topics

Keywords

sequence modeling (1) batch normalization (1) image recognition (1) state-space model (1) recurrent neural network (1) long sequence modeling (1) residual network (1) parallel training (1) linear recurrent unit (1) long range arena (1) adaptive gradient clipping (1) normalizer-free network (1)

Papers

Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues ICML 2024

Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation ICLR 2023

Resurrecting Recurrent Neural Networks for Long Sequences ICML 2023

On the Origin of Implicit Regularization in Stochastic Gradient Descent ICLR 2021

Characterizing signal propagation to close the performance gap in unnormalized ResNets ICLR 2021

High-Performance Large-Scale Image Recognition Without Normalization ICML 2021