← Back to papers
2022 NAACL NAACL 2022

BAD-X: Bilingual Adapters Improve Zero-Shot Cross-Lingual Transfer

Abstract

AbstractAdapter modules enable modular and efficient zero-shot cross-lingual transfer, where current state-of-the-art adapter-based approaches learn specialized language adapters (LAs) for individual languages. In this work, we show that it is more effective to learn bilingual language pair adapters (BAs) when the goal is to optimize performance for a particular source-target transfer direction. Our novel BAD-X adapter framework trades off some modularity of dedicated LAs for improved transfer performance: we demonstrate consistent gains in three standard downstream tasks, and for the majority of evaluated low-resource languages.

🌉 Interdisciplinary Bridge — Deep Learning and Machine Learning and Natural Language Processing
🧭 Keyword Pioneer — bilingual adapter
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Marinela Parovic , Goran Glavaš , Ivan Vulić , Anna Korhonen

Topics

Machine Learning > Learning Types > Zero-Shot Learning Deep Learning > Architectures > Transformers Machine Learning > Learning Types > Transfer Learning Natural Language Processing > Resources & Methods > Transfer Learning

Keywords

zero-shot learning cross-lingual transfer low-resource language adapter module language adapter zero-shot cross-lingual transfer bilingual adapter parameter-efficient transfer
Download PDF

Related papers

EVI: Multilingual Spoken Dialogue Tasks and Dataset for Knowledge-Based Enrolment, Verification, and Identification 2022
Generating Complement Data for Aspect Term Extraction with GPT-2 2022
Regularized Training of Nearest Neighbor Language Models 2022
Systematicity Emerges in Transformers when Abstract Grammatical Roles Guide Attention 2022
Neural Retriever and Go Beyond: A Thesis Proposal 2022