2020
EMNLP
EMNLP 2020
Monolingual Adapters for Zero-Shot Neural Machine Translation
Abstract
AbstractWe propose a novel adapter layer formalism for adapting multilingual models. They are more parameter-efficient than existing adapter layers while obtaining as good or better performance. The layers are specific to one language (as opposed to bilingual adapters) allowing to compose them and generalize to unseen language-pairs. In this zero-shot setting, they obtain a median improvement of +2.77 BLEU points over a strong 20-language multilingual Transformer baseline trained on TED talks.
🌉
Interdisciplinary Bridge
— Deep Learning and Machine Learning and Natural Language Processing
🧭
Keyword Pioneer
— parameter-efficient transfer
🐣
Hot Topic Early Bird
— multilingual transformer
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Speech & Audio
Authors
Topics
Machine Learning > Learning Types > Zero-Shot Learning
Natural Language Processing > Applications > Machine Translation
Machine Learning > Learning Types > Transfer Learning
Natural Language Processing > Generation > Machine Translation
Deep Learning > Optimization & Theory > Model Compression
Deep Learning > Learning Types > Transfer Learning