← Back to papers
2022 ACL ACL 2022

Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation?

Abstract

AbstractWhat can pre-trained multilingual sequence-to-sequence models like mBART contribute to translating low-resource languages? We conduct a thorough empirical experiment in 10 languages to ascertain this, considering five factors: (1) the amount of fine-tuning data, (2) the noise in the fine-tuning data, (3) the amount of pre-training data in the model, (4) the impact of domain mismatch, and (5) language typology. In addition to yielding several heuristics, the experiments form a framework for evaluating the data sensitivities of machine translation systems. While mBART is robust to domain differences, its translations for unseen and typologically distant languages remain below 3.0 BLEU. In answer to our title’s question, mBART is not a low-resource panacea; we therefore encourage shifting the emphasis from new models to new data.

❓ The Questioner
🌉 Interdisciplinary Bridge — Deep Learning and Machine Learning and Natural Language Processing
🧭 Keyword Pioneer — fine-tuning datum
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

En-Shiun Annie Lee , Sarubi Thillainathan , Shravan Nayak , Surangika Ranathunga , David Ifeoluwa Adelani , Ruisi Su , Arya D. McCarthy

Topics

Deep Learning > Techniques > Pretraining Natural Language Processing > Applications > Machine Translation Natural Language Processing > Resources & Methods > Multilingual NLP Machine Learning > Learning Types > Transfer Learning Natural Language Processing > Generation > Machine Translation Machine Learning > Learning Types > Fine-Tuning Deep Learning > Models > Language Models

Keywords

machine translation multilingual machine translation low-resource language pre-trained model sequence-to-sequence model multilingual model fine-tuning datum
Download PDF

Related papers

HIE-SQL: History Information Enhanced Network for Context-Dependent Text-to-SQL Semantic Parsing 2022
KG-CRuSE: Recurrent Walks over Knowledge Graph for Explainable Conversation Reasoning using Semantic Embeddings 2022
Toward Knowledge-Enriched Conversational Recommendation Systems 2022
Investigating the Medical Coverage of a Translation System into Pictographs for Patients with an Intellectual Disability 2022
TableFormer: Robust Transformer Modeling for Table-Text Encoding 2022