← Authors

Dan Alistarh

64 papers · 2015–2026 · 11 conferences · across top CS/AI conferences

Achievements

πŸƒ Academic Marathon (10) πŸŒ‰ Interdisciplinary Bridge 🐝 Cross-Pollinator (8) 🌈 Renaissance Researcher (7) πŸ—ΊοΈ Taxonomy Completionist (11) 🧭 Keyword Pioneer 🐣 Hot Topic Early Bird 🌍 Conference Polyglot (10) 🏠 Conference Loyalist (21) πŸ† Grand Slam πŸ”¬ Deep Specialist (18) πŸ‘‘ Triple Crown 🀝 Dynamic Duo (15) πŸ† Keyword Champion πŸ—ƒοΈ Keyword Collector (174) ❓ The Questioner (2) ⚑ Prolific Year (14) πŸš€ Conference Pioneer πŸ’Ž Century Club (63) πŸ”₯ Unstoppable (9) πŸ“ˆ Trend Setter

Conferences

NIPS (21) ICML (17) ICLR (11) AAAI (3) EMNLP (3) AISTATS (2) CVPR (2) JMLR (2) ACL (1) EACL (1) NAACL (1)

Top co-authors

Elias Frantar (15) Eldar Kurtic (11) Denis Kuznedelev (7) Torsten Hoefler (6) Alexandra Peste (6) Ilia Markov (6) Eugenia Iofinova (5) Mher Safaryan (5) Saleh Ashkboos (5) Ionut-Vlad Modoranu (4)

Research topics

Optimization & Theory (1)

Keywords

model compression (17) stochastic gradient descent (9) distributed learning (7) distributed optimization (7) large language model (6) neural network pruning (5) neural network optimization (5) distributed training (5) model pruning (4) convolutional neural network (4) model quantization (4) second-order optimization (4) neural network compression (4) communication efficiency (4) convergence guarantee (4) gradient quantization (4) non-convex optimization (3) neural network training (3) knowledge distillation (3) convergence analysis (3)

Papers

Speculative Decoding Speed-of-Light: Optimal Lower Bounds via Branching Random Walks EACL 2026
HIGGS: Pushing the Limits of Large Language Model Quantization via the Linearity Theorem NAACL 2025
EvoPress: Accurate Dynamic Model Compression via Evolutionary Search ICML 2025
Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models ICML 2025
QuEST: Stable Training of LLMs with 1-Bit Weights and Activations ICML 2025
Layer-wise Quantization for Quantized Optimistic Dual Averaging ICML 2025
LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics ICLR 2025
The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws ICLR 2025
Scalable Mechanistic Neural Networks ICLR 2025
Wasserstein Distances, Neuronal Entanglement, and Sparsity ICLR 2025
Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence AAAI 2025
β€œGive Me BF16 or Give Me Death”? Accuracy-Performance Trade-Offs in LLM Quantization ACL 2025
SPADE: Sparsity-Guided Debugging for Deep Neural Networks ICML 2024
MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence NIPS 2024
PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression NIPS 2024
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs NIPS 2024
The Iterative Optimal Brain Surgeon: Faster Sparse Recovery by Leveraging Second-Order Information NIPS 2024
AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithms AISTATS 2024
Communication-Efficient Federated Learning With Data and Client Heterogeneity AISTATS 2024
QUIK: Towards End-to-end 4-Bit Inference on Generative Large Language Models EMNLP 2024
Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on Large Language Models EMNLP 2024
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression ICLR 2024
Scaling Laws for Sparsely-Connected Foundation Models ICLR 2024
Extreme Compression of Large Language Models via Additive Quantization ICML 2024
Error Feedback Can Accurately Compress Preconditioners ICML 2024
RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation ICML 2024
OPTQ: Accurate Quantization for Generative Pre-trained Transformers ICLR 2023
CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Models NIPS 2023
ZipLM: Inference-Aware Structured Pruning of Language Models NIPS 2023
Knowledge Distillation Performs Partial Variance Reduction NIPS 2023
SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks at the Edge ICML 2023
Quantized Distributed Training of Large Models with Convergence Guarantees ICML 2023
SparseGPT: Massive Language Models Can be Accurately Pruned in One-Shot ICML 2023
Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures CVPR 2023
CrAM: A Compression-Aware Minimizer ICLR 2023
Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning NIPS 2022
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models EMNLP 2022
How Well Do Sparse ImageNet Models Transfer? CVPR 2022
SPDY: Accurate Pruning with Speedup Guarantees ICML 2022
Towards Tight Communication Lower Bounds for Distributed Optimisation NIPS 2021
Asynchronous Decentralized SGD with Quantized and Local Updates NIPS 2021
Distributed Principal Component Analysis with Limited Communication NIPS 2021
New Bounds For Distributed Mean Estimation and Variance Reduction ICLR 2021
Asynchronous Optimization Methods for Efficient Training of Deep Neural Networks with Guarantees AAAI 2021
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks JMLR 2021
AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks NIPS 2021
M-FAC: Efficient Matrix-Free Approximations of Second-Order Information NIPS 2021
Communication-Efficient Distributed Optimization with Quantized Preconditioners ICML 2021
Byzantine-Resilient Non-Convex Stochastic Gradient Descent ICLR 2021
Elastic Consistency: A Practical Consistency Model for Distributed Stochastic Gradient Descent AAAI 2021
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization JMLR 2021
WoodFisher: Efficient Second-Order Approximation for Neural Network Compression NIPS 2020
Adaptive Gradient Quantization for Data-Parallel SGD NIPS 2020
Scalable Belief Propagation via Relaxed Scheduling NIPS 2020
On the Sample Complexity of Adversarial Multi-Source PAC Learning ICML 2020
Inducing and Exploiting Activation Sparsity for Fast Inference on Deep Neural Networks ICML 2020
Powerset Convolutional Neural Networks NIPS 2019
Distributed Learning over Unreliable Networks ICML 2019
Byzantine Stochastic Gradient Descent NIPS 2018
The Convergence of Sparsified Gradient Methods NIPS 2018
Model compression via distillation and quantization ICLR 2018
QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding NIPS 2017
ZipML: Training Linear Models with End-to-End Low Precision, and a Little Bit of Deep Learning ICML 2017
Streaming Min-max Hypergraph Partitioning NIPS 2015