2018
ACL
ACL 2018
Marian: Cost-effective High-Quality Neural Machine Translation in C++
Abstract
AbstractThis paper describes the submissions of the βMarianβ team to the WNMT 2018 shared task. We investigate combinations of teacher-student training, low-precision matrix products, auto-tuning and other methods to optimize the Transformer model on GPU and CPU. By further integrating these methods with the new averaging attention networks, a recently introduced faster Transformer variant, we create a number of high-quality, high-performance models on the GPU and CPU, dominating the Pareto frontier for this shared task.
π
Interdisciplinary Bridge
β Artificial Intelligence and Machine Learning and Natural Language Processing
π
Trend Setter
β Model Compression
π§
Keyword Pioneer
β transformer model
π£
Hot Topic Early Bird
β neural machine translation
π
Cross-Pollinator
β Artificial Intelligence, Computer Vision, Deep Learning, Healthcare & Medicine, Interdisciplinary, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Speech & Audio
Authors
Topics
Artificial Intelligence > Core AI > Model Compression
Machine Learning > Application Areas > Efficient Computing
Natural Language Processing > Applications > Machine Translation
Natural Language Processing > Generation > Machine Translation
Deep Learning > Optimization & Theory > Model Compression
Deep Learning > Techniques > Knowledge Distillation
Deep Learning > Optimization & Theory > Efficient Computing