2020
EMNLP
EMNLP 2020
Low-Resource Translation as Language Modeling
Abstract
AbstractWe present our submission to the very low resource supervised machine translation task at WMT20. We use a decoder-only transformer architecture and formulate the translation task as language modeling. To address the low-resource aspect of the problem, we pretrain over a similar language parallel corpus. Then, we employ an intermediate back-translation step before fine-tuning. Finally, we present an analysis of the system’s performance.
🌉
Interdisciplinary Bridge
— Deep Learning and Natural Language Processing
🧭
Keyword Pioneer
— decoder-only transformer
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Security & Privacy, Speech & Audio
Authors
Topics
Deep Learning > Architectures > Transformers
Natural Language Processing > Generation > Language Modeling
Natural Language Processing > Applications > Machine Translation
Natural Language Processing > Generation > Machine Translation
Deep Learning > Models > Transformers
Deep Learning > Learning Types > Transfer Learning
Deep Learning > Models > Language Modeling