2021
ACL
ACL 2021
RG PA at SemEval-2021 Task 1: A Contextual Attention-based Model with RoBERTa for Lexical Complexity Prediction
Abstract
AbstractIn this paper we propose a contextual attention based model with two-stage fine-tune training using RoBERTa. First, we perform the first-stage fine-tune on corpus with RoBERTa, so that the model can learn some prior domain knowledge. Then we get the contextual embedding of context words based on the token-level embedding with the fine-tuned model. And we use Kfold cross-validation to get K models and ensemble them to get the final result. Finally, we attain the 2nd place in the final evaluation phase of sub-task 2 with pearson correlation of 0.8575.
🧭
Keyword Pioneer
— lexical complexity
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Vision, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Speech & Audio
🌉
Interdisciplinary Bridge
— Artificial Intelligence and Machine Learning and Natural Language Processing
Authors
Topics
Artificial Intelligence > Core AI > Multimodal Learning
Artificial Intelligence > Learning Paradigms > Transfer Learning
Machine Learning > Core Methods > Regression
Machine Learning > Core Methods > Representation Learning
Machine Learning > Optimization & Theory > Neural Network Optimization
Natural Language Processing > Understanding > Semantic Analysis
Natural Language Processing > Applications > Text Classification
Machine Learning > Learning Paradigms > Transfer Learning
Machine Learning > Learning Types > Transfer Learning