← Learning Types
Machine Learning › Learning Types ›

Knowledge Distillation

503 directly classified papers

Papers per year

Papers

Team ISM at CLPsych 2024: Extracting Evidence of Suicide Risk from Reddit Posts with Knowledge Self-Generation and Output Refinement using A Large Language Model EACL 2024
M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation ACL 2024
KNN-Instruct: Automatic Instruction Construction with K Nearest Neighbor Deduction EMNLP 2024
Archimedes-AUEB at SemEval-2024 Task 5: LLM explains Civil Procedure SEMEVAL 2024
Self-Supervised Quantization-Aware Knowledge Distillation AISTATS 2024
Deep Classifier Mimicry without Data Access AISTATS 2024
Text Simplification via Adaptive Teaching ACL 2024
BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation ACL 2024
Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration NIPS 2024
Contextual Affinity Distillation for Image Anomaly Detection WACV 2024
Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning ACL 2024
A Kernel Perspective on Distillation-based Collaborative Learning NIPS 2024
A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis NIPS 2024
Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learner NIPS 2024
Decoupled Kullback-Leibler Divergence Loss NIPS 2024
CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning ACL 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning NIPS 2024
Wasserstein Distance Rivals Kullback-Leibler Divergence for Knowledge Distillation NIPS 2024
The ALCHEmist: Automated Labeling 500x CHEaper than LLM Data Annotators NIPS 2024
An Empirical Study of Multilingual Reasoning Distillation for Question Answering EMNLP 2024
Leros: Learning Explicit Reasoning on Synthesized Data for Commonsense Question Answering COLING 2024
The Mamba in the Llama: Distilling and Accelerating Hybrid Models NIPS 2024
LaKD: Length-agnostic Knowledge Distillation for Trajectory Prediction with Any Length Observations NIPS 2024
Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic EMNLP 2024
BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts NIPS 2024