← Back to papers
2020 AAAI AAAI 2020

Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation

Abstract

Abstract The recently proposed massively multilingual neural machine translation (NMT) system has been shown to be capable of translating over 100 languages to and from English within a single model (Aharoni, Johnson, and Firat 2019). Its improved translation performance on low resource languages hints at potential cross-lingual transfer capability for downstream tasks. In this paper, we evaluate the cross-lingual effectiveness of representations from the encoder of a massively multilingual NMT model on 5 downstream classification and sequence labeling tasks covering a diverse set of over 50 languages. We compare against a strong baseline, multilingual BERT (mBERT) (Devlin et al. 2018), in different cross-lingual transfer learning scenarios and show gains in zero-shot transfer in 4 out of these 5 tasks.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Deep Learning and Natural Language Processing
🐣 Hot Topic Early Bird — zero-shot transfer
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Aditya Siddhant , Melvin Johnson , Henry Tsai , Naveen Ari , Jason Riesa , Ankur Bapna , Orhan Firat , Karthik Raman

Topics

Artificial Intelligence > Learning Paradigms > Transfer Learning Natural Language Processing > Resources & Methods > Multilingual NLP Natural Language Processing > Generation > Machine Translation Deep Learning > Learning Types > Transfer Learning

Keywords

sequence labeling text classification cross-lingual transfer zero-shot transfer multilingual neural machine translation encoder representation
Download PDF

Related papers

Enhancing Pointer Network for Sentence Ordering with Pairwise Ordering Predictions 2020
CopyMTL: Copy Mechanism for Joint Extraction of Entities and Relations with Multi-Task Learning 2020
Neural Simile Recognition with Cyclic Multitask Learning and Local Attention 2020
Being Optimistic to Be Conservative: Quickly Learning a CVaR Policy 2020
Multi-Point Semantic Representation for Intent Classification 2020