2021
EMNLP
EMNLP 2021
Papago’s Submission for the WMT21 Quality Estimation Shared Task
Abstract
AbstractThis paper describes Papago submission to the WMT 2021 Quality Estimation Task 1: Sentence-level Direct Assessment. Our multilingual Quality Estimation system explores the combination of Pretrained Language Models and Multi-task Learning architectures. We propose an iterative training pipeline based on pretraining with large amounts of in-domain synthetic data and finetuning with gold (labeled) data. We then compress our system via knowledge distillation in order to reduce parameters yet maintain strong performance. Our submitted multilingual systems perform competitively in multilingual and all 11 individual language pair settings including zero-shot.
🌉
Interdisciplinary Bridge
— Deep Learning and Machine Learning and Natural Language Processing
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
Authors
Topics
Machine Learning > Application Areas > Knowledge Distillation
Deep Learning > Techniques > Pretraining
Natural Language Processing > Applications > Machine Translation
Machine Learning > Application Areas > Model Compression
Machine Learning > Learning Types > Knowledge Distillation
Deep Learning > Learning Types > Transfer Learning
Deep Learning > Learning Types > Multi-Task Learning