← Learning Types
Machine Learning › Learning Types ›

Knowledge Distillation

503 directly classified papers

Papers per year

Papers

The Importance of Being Parameters: An Intra-Distillation Method for Serious Gains EMNLP 2022
Production federated keyword spotting via distillation, filtering, and joint federated-centralized training INTERSPEECH 2022
Referee: Reference-Free Sentence Summarization with Sharper Controllability through Symbolic Knowledge Distillation EMNLP 2022
Distillation-Resistant Watermarking for Model Protection in NLP EMNLP 2022
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation NAACL 2022
KroneckerBERT: Significant Compression of Pre-trained Language Models Through Kronecker Decomposition and Knowledge Distillation NAACL 2022
Collective Relevance Labeling for Passage Retrieval NAACL 2022
Causal Distillation for Language Models NAACL 2022
Knowledge Distillation: Bad Models Can Be Good Role Models NIPS 2022
EyePAD++: A Distillation-Based Approach for Joint Eye Authentication and Presentation Attack Detection Using Periocular Images CVPR 2022
How Many Observations Are Enough? Knowledge Distillation for Trajectory Forecasting CVPR 2022
Performance-Aware Mutual Knowledge Distillation for Improving Neural Architecture Search CVPR 2022
Learning Cross-Lingual IR from an English Retriever NAACL 2022
Nearest Neighbor Knowledge Distillation for Neural Machine Translation NAACL 2022
Learning to Generate Question by Asking Question: A Primal-Dual Approach with Uncommon Word Generation EMNLP 2022
Transferring Knowledge from Structure-aware Self-attention Language Model to Sequence-to-Sequence Semantic Parsing COLING 2022
Generic Overgeneralization in Pre-trained Language Models COLING 2022
CILDA: Contrastive Data Augmentation Using Intermediate Layer Knowledge Distillation COLING 2022
Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher COLING 2022
Enhancing Self-Attention with Knowledge-Assisted Attention Maps NAACL 2022
Diversifying Neural Dialogue Generation via Negative Distillation NAACL 2022
Knowledge Inheritance for Pre-trained Language Models NAACL 2022
TAKE: Topic-shift Aware Knowledge sElection for Dialogue Generation COLING 2022
Learning Multiple Adverse Weather Removal via Two-Stage Knowledge Learning and Multi-Contrastive Regularization: Toward a Unified Model CVPR 2022
From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models EMNLP 2022