← Learning Types
Deep Learning › Learning Types ›

Knowledge Distillation

790 directly classified papers

Papers per year

Papers

LaKD: Length-agnostic Knowledge Distillation for Trajectory Prediction with Any Length Observations NIPS 2024
Self-Distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approach AAAI 2024
FAKD: Feature Augmented Knowledge Distillation for Semantic Segmentation WACV 2024
Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion AAAI 2024
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion NIPS 2024
Memory-Augmenting Decoder-Only Language Models through Encoders (Student Abstract) AAAI 2024
Turning Dust into Gold: Distilling Complex Reasoning Capabilities from LLMs by Leveraging Negative Data AAAI 2024
Boosting Residual Networks with Group Knowledge AAAI 2024
Data Distribution Distilled Generative Model for Generalized Zero-Shot Recognition AAAI 2024
Let All Be Whitened: Multi-Teacher Distillation for Efficient Visual Retrieval AAAI 2024
A Privacy-preserving Approach to Ingest Knowledge from Proprietary Web-based to Locally Run Models for Medical Progress Note Generation ACL 2024
Understanding the Role of the Projector in Knowledge Distillation AAAI 2024
Progressively Knowledge Distillation via Re-parameterizing Diffusion Reverse Process AAAI 2024
Incremental pre-training from smaller language models ACL 2024
A Kernel Perspective on Distillation-based Collaborative Learning NIPS 2024
Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging NIPS 2024
LSTKC: Long Short-Term Knowledge Consolidation for Lifelong Person Re-identification AAAI 2024
The Mamba in the Llama: Distilling and Accelerating Hybrid Models NIPS 2024
BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts NIPS 2024
MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts AAAI 2024
ADAM: Dense Retrieval Distillation with Adaptive Dark Examples ACL 2024
Improved Distribution Matching Distillation for Fast Image Synthesis NIPS 2024
Simple and Fast Distillation of Diffusion Models NIPS 2024
Federated Learning with Extremely Noisy Clients via Negative Distillation AAAI 2024
Probabilistic Conformal Distillation for Enhancing Missing Modality Robustness NIPS 2024