Artificial Intelligence › Core AI ›

Knowledge Distillation

99 directly classified papers

Papers per year

Papers

Leveraging Entity Information for Cross-Modality Correlation Learning: The Entity-Guided Multimodal Summarization ACL 2024

ADAM: Dense Retrieval Distillation with Adaptive Dark Examples ACL 2024

A Privacy-preserving Approach to Ingest Knowledge from Proprietary Web-based to Locally Run Models for Medical Progress Note Generation ACL 2024

LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement EMNLP 2024

Question-guided Knowledge Graph Re-scoring and Injection for Knowledge Graph Question Answering EMNLP 2024

PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning EMNLP 2024

BiKT: Enabling Bidirectional Knowledge Transfer Between Pretrained Models and Sequential Downstream Tasks EMNLP 2024

Divide-or-Conquer? Which Part Should You Distill Your LLM? EMNLP 2024

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging NIPS 2024

An LLM-Enhanced Adversarial Editing System for Lexical Simplification COLING 2024

FIRST: Teach A Reliable Large Language Model Through Efficient Trustworthy Distillation EMNLP 2024

LEAD: Exploring Logit Space Evolution for Model Selection CVPR 2024

Active Object Detection with Knowledge Aggregation and Distillation from Large Models CVPR 2024

X-Adapter: Adding Universal Compatibility of Plugins for Upgraded Diffusion Model CVPR 2024

InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance EMNLP 2024

Less is More: Parameter-Efficient Selection of Intermediate Tasks for Transfer Learning EMNLP 2024

An Empirical Study of Multilingual Reasoning Distillation for Question Answering EMNLP 2024

Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarios ACL 2024

How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives ACL 2023

Distilling Efficient Language-Specific Models for Cross-Lingual Transfer ACL 2023

Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method ACL 2023

Learning from a Friend: Improving Event Extraction via Self-Training with Feedback from Abstract Meaning Representation ACL 2023

Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference ACL 2023

A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models ACL 2023

Learn From One Specialized Sub-Teacher: One-to-One Mapping for Feature-Based Knowledge Distillation EMNLP 2023