← Back to papers
2021 EMNLP EMNLP 2021

The IICT-Yverdon System for the WMT 2021 Unsupervised MT and Very Low Resource Supervised MT Task

Abstract

AbstractIn this paper, we present the systems submitted by our team from the Institute of ICT (HEIG-VD / HES-SO) to the Unsupervised MT and Very Low Resource Supervised MT task. We first study the improvements brought to a baseline system by techniques such as back-translation and initialization from a parent model. We find that both techniques are beneficial and suffice to reach performance that compares with more sophisticated systems from the 2020 task. We then present the application of this system to the 2021 task for low-resource supervised Upper Sorbian (HSB) to German translation, in both directions. Finally, we present a contrastive system for HSB-DE in both directions, and for unsupervised German to Lower Sorbian (DSB) translation, which uses multi-task training with various training schedules to improve over the baseline.

🌉 Interdisciplinary Bridge — Deep Learning and Machine Learning and Natural Language Processing
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Speech & Audio

Authors

Àlex R. Atrio , Gabriel Luthier , Axel Fahy , Giorgos Vernikos , Andrei Popescu-Belis , Ljiljana Dolamic

Topics

Machine Learning > Learning Types > Unsupervised Learning Natural Language Processing > Applications > Machine Translation Machine Learning > Learning Types > Transfer Learning Natural Language Processing > Generation > Machine Translation Deep Learning > Learning Types > Transfer Learning Deep Learning > Learning Types > Unsupervised Learning

Keywords

neural machine translation low-resource language model initialization unsupervised machine translation multi-task training
Download PDF

Related papers

Continual Learning in Multilingual NMT via Language-Specific Embeddings 2021
MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents 2021
Efficient Multi-Task Auxiliary Learning: Selecting Auxiliary Data by Feature Similarity 2021
Neural Machine Translation with Heterogeneous Topic Knowledge Embeddings 2021
Semantics-Preserved Data Augmentation for Aspect-Based Sentiment Analysis 2021