← Authors

Ali Ghodsi

25 papers · 2011–2024 · 10 conferences · across top CS/AI conferences

Achievements

🐣 Hot Topic Early Bird πŸŒ‰ Interdisciplinary Bridge 🧭 Keyword Pioneer 🌍 Conference Polyglot (10) πŸƒ Academic Marathon (13) 🐝 Cross-Pollinator (12) 🌈 Renaissance Researcher (5) πŸ—ΊοΈ Taxonomy Completionist (48) πŸ”¬ Deep Specialist (11) 🀝 Dynamic Duo (17) ⚑ Prolific Year (5) πŸš€ Conference Pioneer πŸ“ˆ Trend Setter πŸ—ƒοΈ Keyword Collector (83) πŸ’Ž Century Club (25) ❓ The Questioner (2)

Conferences

EMNLP (7) EACL (4) NSDI (4) ACL (2) AISTATS (2) NAACL (2) COLING (1) IJCNLP (1) MIDL (1) NIPS (1)

Top co-authors

Mehdi Rezagholizadeh (17) Aref Jafari (6) Ahmad Rashid (5) Ion Stoica (4) Ivan Kobyzev (4) Pranav Sharma (3) Boxing Chen (3) Abbas Ghaddar (3) Ehsan Kamalloo (3) Peng Lu (3)

Keywords

knowledge distillation (12) model compression (11) natural language understanding (3) transfer learning (3) pre-trained language model (3) large language model (3) neural network optimization (2) attention mechanism (2) representation learning (2) capacity gap (2) parameter-efficient fine-tuning (2) intermediate layer (2) low-rank adaptation (2) neural network (2) knowledge transfer (1) sample efficiency (1) matrix factorization (1) sequence modeling (1) transformer architecture (1) variational inference (1)

Papers

Efficient Citer: Tuning Large Language Models for Enhanced Answer Quality and Verification NAACL 2024
Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling NIPS 2024
QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning EMNLP 2024
Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference EACL 2024
Do we need Label Regularization to Fine-tune Pre-trained Language Models? EACL 2023
DyLoRA: Parameter-Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation EACL 2023
When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation ACL 2022
Continuation KD: Improved Knowledge Distillation through the Lens of Continuation Optimization EMNLP 2022
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging EMNLP 2022
Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher COLING 2022
KroneckerBERT: Significant Compression of Pre-trained Language Models Through Kronecker Decomposition and Knowledge Distillation NAACL 2022
RW-KD: Sample-wise Loss Terms Re-Weighting for Knowledge Distillation EMNLP 2021
Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax ACL 2021
Annealing Knowledge Distillation EACL 2021
Universal-KD: Attention-based Output-Grounded Intermediate Layer Knowledge Distillation EMNLP 2021
How to Select One Among All ? An Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding EMNLP 2021
Knowledge Distillation with Noisy Labels for Natural Language Understanding EMNLP 2021
Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax IJCNLP 2021
CNN and Deep Sets for End-to-End Whole Slide Image Representation Learning MIDL 2021
Robust Locally-Linear Controllable Embedding AISTATS 2018
FairRide: Near-Optimal, Fair Cache Sharing NSDI 2016
HUG: Multi-Resource Fairness for Correlated and Elastic Demands NSDI 2016
Effective Straggler Mitigation: Attack of the Clones NSDI 2013
PACMan: Coordinated Memory Caching for Parallel Jobs NSDI 2012
A novel greedy algorithm for NystrΓΆm approximation AISTATS 2011