← Back to papers
2019 ACL ACL 2019

Target Conditioned Sampling: Optimizing Data Selection for Multilingual Neural Machine Translation

Abstract

AbstractTo improve low-resource Neural Machine Translation (NMT) with multilingual corpus, training on the most related high-resource language only is generally more effective than us- ing all data available (Neubig and Hu, 2018). However, it remains a question whether a smart data selection strategy can further improve low-resource NMT with data from other auxiliary languages. In this paper, we seek to construct a sampling distribution over all multilingual data, so that it minimizes the training loss of the low-resource language. Based on this formulation, we propose and efficient algorithm, (TCS), which first samples a target sentence, and then conditionally samples its source sentence. Experiments show TCS brings significant gains of up to 2 BLEU improvements on three of four languages we test, with minimal training overhead.

🌉 Interdisciplinary Bridge — Machine Learning and Natural Language Processing
🐝 Cross-Pollinator — Artificial Intelligence, Deep Learning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Speech & Audio
🧭 Keyword Pioneer — target conditioned sampling
🐣 Hot Topic Early Bird — data selection

Authors

Xinyi Wang , Graham Neubig

Topics

Machine Learning > Optimization & Theory > Optimization Machine Learning > Application Areas > Data Augmentation Natural Language Processing > Applications > Machine Translation Natural Language Processing > Generation > Machine Translation Machine Learning > Learning Types > Multi-Modal Learning Artificial Intelligence > Core AI > Natural Language Processing Machine Learning > Learning Types > Multi-Lingual Learning Deep Learning > Learning Types > Multi-Lingual Learning

Keywords

neural machine translation sampling distribution low-resource language data selection low-resource translation multilingual training multilingual neural machine translation target conditioned sampling
Download PDF

Related papers

What do phone embeddings learn about Phonology? 2019
Unsupervised Morphological Segmentation for Low-Resource Polysynthetic Languages 2019
Understanding Undesirable Word Embedding Associations 2019
Inferential Machine Comprehension: Answering Questions by Recursively Deducing the Evidence Chain from Text 2019
Domain Adaptation of Neural Machine Translation by Lexicon Induction 2019