← Back to papers
2022 EMNLP EMNLP 2022

HW-TSC Systems for WMT22 Very Low Resource Supervised MT Task

Abstract

AbstractThis paper describes the submissions of Huawei translation services center (HW-TSC) to the WMT22 Very Low Resource Supervised MT task. We participate in all 6 supervised tracks including all combinations between Upper/Lower Sorbian (Hsb/Dsb) and German (De). Our systems are build on deep Transformer with a large filter size. We use multilingual transfer with German-Czech (De-Cs) and German-Polish (De-Pl) parallel data. We also utilize regularized dropout (R-Drop), back translation, fine-tuning and ensemble to improve the system performance. According to the official evaluation results on OCELoT, our supervised systems of all 6 language directions get the highest BLEU scores among all submissions. Our pre-trained multilingual model for unsupervised De2Dsb and Dsb2De translation also gain highest BLEU.

🌉 Interdisciplinary Bridge — Deep Learning and Machine Learning and Natural Language Processing
🧭 Keyword Pioneer — regularized dropout
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Shaojun Li , Yuanchang Luo , Daimeng Wei , Zongyao Li , Hengchao Shang , Xiaoyu Chen , Zhanglin Wu , Jinlong Yang , Zhiqiang Rao , Zhengzhe Yu , Yuhao Xie , Lizhi Lei , Hao Yang , Ying Qin

Topics

Machine Learning > Application Areas > Domain Adaptation Deep Learning > Architectures > Transformers Natural Language Processing > Applications > Machine Translation Machine Learning > Learning Types > Transfer Learning Natural Language Processing > Generation > Machine Translation Deep Learning > Models > Transformers Deep Learning > Learning Types > Representation Learning

Keywords

ensemble learning neural machine translation low-resource translation back translation regularized dropout multilingual transfer deep transformer
Download PDF

Related papers

Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport 2022
Generative Entity Typing with Curriculum Learning 2022
Towards Reinterpreting Neural Topic Models via Composite Activations 2022
Weakly Supervised Headline Dependency Parsing 2022
Cross-modal Transfer Between Vision and Language for Protest Detection 2022