← Authors

Surya Ganguli

47 papers · 2010–2025 · 5 conferences · across top CS/AI conferences

Achievements

🌍 Conference Polyglot (5) 🐣 Hot Topic Early Bird 🌈 Renaissance Researcher (7) πŸŒ‰ Interdisciplinary Bridge 🐝 Cross-Pollinator (13) πŸ—ΊοΈ Taxonomy Completionist (12) πŸƒ Academic Marathon (15) 🧭 Keyword Pioneer 🌟 Keyword Trendsetter Combo (4) 🏠 Conference Loyalist (28) πŸ† Keyword Champion πŸ‘‘ Triple Crown πŸ”¬ Deep Specialist (18) ⚑ Prolific Year (6) πŸ—ƒοΈ Keyword Collector (192) πŸ’Ž Century Club (47) ❓ The Questioner πŸ”₯ Unstoppable (13) πŸ“ˆ Trend Setter

Conferences

NIPS (28) ICML (9) ICLR (8) AISTATS (1) EMNLP (1)

Top co-authors

Daniel Kunin (6) Aran Nayebi (6) Jascha Sohl-dickstein (5) Mansheej Paul (5) Niru Maheswaranathan (5) Gintare Karolina Dziugaite (4) Feng Chen (4) Ben Poole (4) Ben Sorscher (3) David Sussillo (3)

Research topics

Education (1)

Keywords

neural network (6) recurrent neural network (6) dynamical system (4) deep neural network (4) phase transition (3) convolutional neural network (3) training dynamics (2) stochastic gradient descent (2) sparse training (2) representation learning (2) neural response (2) synaptic memory (2) feature learning (2) lottery ticket hypothesis (2) sample complexity (2) neural encoding (2) gradient descent (2) neural dynamics (2) loss landscape (2) generative model (2)

Papers

Features are fate: a theory of transfer learning in high-dimensional regression ICML 2025
An analytic theory of creativity in convolutional diffusion models ICML 2025
Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning NIPS 2024
Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression NIPS 2023
Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks NIPS 2023
Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? ICLR 2023
Disentanglement with Biological Constraints: A Theory of Functional Cell Types ICLR 2023
The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks ICLR 2023
Information Geometry of the Retinal Representation Manifold NIPS 2023
MetaMorph: Learning Universal Controllers with Transformers ICLR 2022
How many degrees of freedom do we need to train deep networks: a loss landscape perspective ICLR 2022
Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks NIPS 2022
Beyond neural scaling laws: beating power law scaling via data pruning NIPS 2022
A theory of high dimensional regression with arbitrary correlations between input features and target functions: sample complexity, multiple descent curves and a hierarchy of phase transitions ICML 2021
Deep Learning on a Data Diet: Finding Important Examples Early in Training NIPS 2021
Understanding self-supervised learning dynamics without contrastive pairs ICML 2021
Explaining heterogeneity in medial entorhinal cortex with task-driven neural networks NIPS 2021
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics ICLR 2021
Pruning neural networks without any data by iteratively conserving synaptic flow NIPS 2020
Identifying Learning Rules From Neural Network Observables NIPS 2020
Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel NIPS 2020
Predictive coding in balanced neural networks with noise, chaos and delays NIPS 2020
RNNs can generate bounded hierarchical languages with optimal memory EMNLP 2020
Two Routes to Scalable Credit Assignment without Weight Symmetry ICML 2020
Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics NIPS 2019
A unified theory for the origin of grid cells through the lens of pattern formation NIPS 2019
Universality and individuality in neural dynamics across large populations of recurrent networks NIPS 2019
A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs ICLR 2019
An analytic theory of generalization dynamics and transfer learning in deep linear networks ICLR 2019
From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction NIPS 2019
The emergence of spectral universality in deep networks AISTATS 2018
Task-Driven Convolutional Recurrent Models of the Visual System NIPS 2018
The emergence of multiple retinal cell types through efficient coding of natural movies NIPS 2018
Statistical mechanics of low-rank tensor decomposition NIPS 2018
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice NIPS 2017
Continual Learning Through Synaptic Intelligence ICML 2017
On the Expressive Power of Deep Neural Networks ICML 2017
Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net NIPS 2017
Deep Learning Models of the Retinal Response to Natural Scenes NIPS 2016
An equivalence between high dimensional Bayes optimal inference and M-estimation NIPS 2016
Exponential expressivity in deep neural networks through transient chaos NIPS 2016
Deep Unsupervised Learning using Nonequilibrium Thermodynamics ICML 2015
Deep Knowledge Tracing NIPS 2015
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization NIPS 2014
Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods ICML 2014
A memory frontier for complex synapses NIPS 2013
Short-term memory in neuronal networks through dynamical compressed sensing NIPS 2010