Frederik Kunstner

8 papers · 2018–2024 · 4 conferences · across top CS/AI conferences

Achievements

🌉 Interdisciplinary Bridge 🏃 Academic Marathon (6) 🐝 Cross-Pollinator (15) 🌍 Conference Polyglot (4) 🧭 Keyword Pioneer 🐣 Hot Topic Early Bird 🔥 Unstoppable (7)

Conferences

NIPS (4) ICLR (2) AISTATS (1) IJCAI (1)

Top co-authors

Mark Schmidt (6) Raunak Kumar (2) Philipp Hennig (2) Lukas Balles (1) Alan Milligan (1) Victor Sanches Portella (1) Nicholas Harvey (1) Aaron Mishkin (1) Felix Dangel (1) Robin Yadav (1)

Keywords

expectation maximization (2) mirror descent (2) natural gradient (2) kl divergence (2) class imbalance (1) fisher information (1) gradient descent (1) exponential family (1) natural gradient descent (1) exponential families (1) fisher information matrix (1) hyperparameter tuning (1) second-order optimization (1) latent variable model (1) convergence rate (1) adam optimizer (1) heavy-tailed distribution (1) language model (1) second-order method (1) uncertainty estimation (1)

Papers

Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models NIPS 2024

Noise Is Not the Main Factor Behind the Gap Between Sgd and Adam on Transformers, But Sign Descent Might Be ICLR 2023

Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking NIPS 2023

Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent (Extended Abstract) IJCAI 2022

Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent AISTATS 2021

BackPACK: Packing more into Backprop ICLR 2020

Limitations of the empirical Fisher approximation for natural gradient descent NIPS 2019

SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient NIPS 2018