2019
EMNLP
EMNLP 2019
Long Warm-up and Self-Training: Training Strategies of NICT-2 NMT System at WAT-2019
Abstract
AbstractThis paper describes the NICT-2 neural machine translation system at the 6th Workshop on Asian Translation. This system employs the standard Transformer model but features the following two characteristics. One is the long warm-up strategy, which performs a longer warm-up of the learning rate at the start of the training than conventional approaches. Another is that the system introduces self-training approaches based on multiple back-translations generated by sampling. We participated in three tasks—ASPEC.en-ja, ASPEC.ja-en, and TDDC.ja-en—using this system.
🌉
Interdisciplinary Bridge
— Deep Learning and Machine Learning and Natural Language Processing
🧭
Keyword Pioneer
— warm-up strategy
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Security & Privacy, Speech & Audio
Authors
Topics
Machine Learning > Learning Types > Self-Supervised Learning
Deep Learning > Architectures > Transformers
Natural Language Processing > Applications > Machine Translation
Natural Language Processing > Generation > Machine Translation
Deep Learning > Models > Transformers
Deep Learning > Learning Types > Machine Translation