2021
EMNLP
EMNLP 2021
Investigating Post-pretraining Representation Alignment for Cross-Lingual Question Answering
Abstract
AbstractHuman knowledge is collectively encoded in the roughly 6500 languages spoken around the world, but it is not distributed equally across languages. Hence, for information-seeking question answering (QA) systems to adequately serve speakers of all languages, they need to operate cross-lingually. In this work we investigate the capabilities of multilingually pretrained language models on cross-lingual QA. We find that explicitly aligning the representations across languages with a post-hoc finetuning step generally leads to improved performance. We additionally investigate the effect of data size as well as the language choice in this fine-tuning step, also releasing a dataset for evaluating cross-lingual QA systems.
🌉
Interdisciplinary Bridge
— Deep Learning and Machine Learning and Natural Language Processing
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
Authors
Topics
Natural Language Processing > Applications > Question Answering
Natural Language Processing > Resources & Methods > Multilingual NLP
Machine Learning > Learning Types > Transfer Learning
Natural Language Processing > Resources & Methods > Language Modeling
Machine Learning > Learning Types > Multi-Modal Learning
Deep Learning > Learning Types > Transfer Learning