← Authors

Jascha Sohl-dickstein

44 papers · 2012–2024 · 5 conferences · across top CS/AI conferences

Achievements

🧭 Keyword Pioneer 🌍 Conference Polyglot (5) πŸƒ Academic Marathon (12) 🐣 Hot Topic Early Bird 🌈 Renaissance Researcher (6) πŸŒ‰ Interdisciplinary Bridge 🐝 Cross-Pollinator (12) πŸ—ΊοΈ Taxonomy Completionist (14) 🌟 Keyword Trendsetter Combo (5) 🀝 Dynamic Duo (11) πŸ‘‘ Triple Crown πŸ”¬ Deep Specialist (12) πŸ† Keyword Champion (2) πŸ“ˆ Trend Setter πŸš€ Conference Pioneer ⚑ Prolific Year (5) πŸ—ƒοΈ Keyword Collector (146) πŸ’Ž Century Club (44) πŸ”₯ Unstoppable (11)

Conferences

ICML (18) NIPS (14) ICLR (10) IJCAI (1) JMLR (1)

Top co-authors

Roman Novak (11) Jeffrey Pennington (10) Jaehoon Lee (8) Luke Metz (8) Lechao Xiao (7) Niru Maheswaranathan (6) Yasaman Bahri (6) Surya Ganguli (5) Ben Poole (4) Jiri Hron (4)

Research topics

Education (1)

Keywords

learned optimizer (5) neural network optimization (5) neural network (4) neural tangent kernel (3) markov chain monte carlo (3) neural network gaussian process (3) generative model (3) stochastic gradient descent (3) gradient descent (3) neural network training (3) batch size (2) recurrent neural network (2) energy-based model (2) evolution strategy (2) variance reduction (2) stochastic process (2) gradient estimation (2) mean field theory (2) batch normalization (2) diffusion model (2)

Papers

Scaling Exponents Across Parameterizations and Optimizers ICML 2024
Small-scale proxies for large-scale Transformer training instabilities ICLR 2024
Position: Levels of AGI for Operationalizing Progress on the Path to AGI ICML 2024
Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies NIPS 2023
Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC ICML 2023
Wide Bayesian neural networks have a simple weight posterior: theory and accelerated sampling ICML 2022
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies (Extended Abstract) IJCAI 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases NIPS 2022
Fast Finite Width Neural Tangent Kernel ICML 2022
Score-Based Generative Modeling through Stochastic Differential Equations ICLR 2021
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies ICML 2021
Whitening and Second Order Optimization Both Make Information in the Dataset Unusable During Training, and Can Reduce or Prevent Generalization ICML 2021
Reverse engineering learned optimizers reveals known and novel mechanisms NIPS 2021
Infinite attention: NNGP and NTK for deep attention networks ICML 2020
Finite Versus Infinite Neural Networks: an Empirical Study NIPS 2020
Neural Tangents: Fast and Easy Infinite Neural Networks in Python ICLR 2020
Your GAN is Secretly an Energy-based Model and You Should Use Discriminator Driven Latent Sampling NIPS 2020
Measuring the Effects of Data Parallelism on Neural Network Training JMLR 2019
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent NIPS 2019
Invertible Convolutional Flow NIPS 2019
A Mean Field Theory of Batch Normalization ICLR 2019
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes ICLR 2019
Meta-Learning Update Rules for Unsupervised Representation Learning ICLR 2019
Adversarial Reprogramming of Neural Networks ICLR 2019
Guided evolutionary strategies: augmenting random search with surrogate gradients ICML 2019
Understanding and correcting pathologies in the training of learned optimizers ICML 2019
The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study ICML 2019
Deep Neural Networks as Gaussian Processes ICLR 2018
Sensitivity and Generalization in Neural Networks: an Empirical Study ICLR 2018
Generalizing Hamiltonian Monte Carlo with Neural Networks ICLR 2018
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks ICML 2018
PCA of high dimensional random walks with comparison to neural network training NIPS 2018
Adversarial Examples that Fool both Computer Vision and Time-Limited Humans NIPS 2018
Input Switched Affine Networks: An RNN Architecture Designed for Interpretability ICML 2017
On the Expressive Power of Deep Neural Networks ICML 2017
Learned Optimizers that Scale and Generalize ICML 2017
REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models NIPS 2017
SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability NIPS 2017
Exponential expressivity in deep neural networks through transient chaos NIPS 2016
Deep Unsupervised Learning using Nonequilibrium Thermodynamics ICML 2015
Deep Knowledge Tracing NIPS 2015
Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods ICML 2014
Hamiltonian Monte Carlo Without Detailed Balance ICML 2014
Training sparse natural image models with a fast Gibbs sampler of an extended state space NIPS 2012