← Learning Types
Machine Learning › Learning Types ›

Knowledge Distillation

503 directly classified papers

Papers per year

Papers

Ensemble Attention Distillation for Privacy-Preserving Federated Learning ICCV 2021
Distilling Virtual Examples for Long-Tailed Recognition ICCV 2021
AlphaNet: Improved Training of Supernets with Alpha-Divergence ICML 2021
RankDistil: Knowledge Distillation for Ranking AISTATS 2021
Weight Distillation: Transferring the Knowledge in Neural Network Parameters ACL 2021
Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates ACL 2021
Question Generation for Adaptive Education ACL 2021
Unsupervised Representation Transfer for Small Networks: I Believe I Can Distill On-the-Fly NIPS 2021
ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques AAAI 2021
Interpretable Embedding Procedure Knowledge Transfer via Stacked Principal Component Analysis and Graph Neural Network AAAI 2021
Collaborative Group Learning AAAI 2021
On Learning the Geodesic Path for Incremental Learning CVPR 2021
Simulating Unknown Target Models for Query-Efficient Black-Box Attacks CVPR 2021
Data-Free Model Extraction CVPR 2021
Students Who Study Together Learn Better: On the Importance of Collective Knowledge Distillation for Domain Transfer in Fact Verification EMNLP 2021
Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answering EMNLP 2021
Collaborative Learning of Bidirectional Decoders for Unsupervised Text Style Transfer EMNLP 2021
MSD: Saliency-aware Knowledge Distillation for Multimodal Understanding EMNLP 2021
NVIDIA NeMo’s Neural Machine Translation Systems for English-German and English-Russian News and Biomedical Tasks at WMT21 EMNLP 2021
Papago’s Submission for the WMT21 Quality Estimation Shared Task EMNLP 2021
Knowledge Distillation with Noisy Labels for Natural Language Understanding EMNLP 2021
MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation ACL 2021
Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor ACL 2021
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking EMNLP 2021
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression EMNLP 2021