2020
COLING
COLING 2020
KAFK at SemEval-2020 Task 12: Checkpoint Ensemble of Transformers for Hate Speech Classification
Abstract
AbstractThis paper presents the approach of Team KAFK for the English edition of SemEval-2020 Task 12. We use checkpoint ensembling to create ensembles of BERT-based transformers and show that it can improve the performance of classification systems. We explore attention mask dropout to mitigate for the poor constructs of social media texts. Our classifiers scored macro-f1 of 0.909, 0.551 and 0.616 for subtasks A, B and C respectively. The code is publicly released online.
🌉
Interdisciplinary Bridge
— Deep Learning and Machine Learning
🧭
Keyword Pioneer
— checkpoint ensembling
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
Authors
Topics
Machine Learning > Core Methods > Classification
Machine Learning > Application Areas > Data Augmentation
Machine Learning > Application Areas > Model Merging
Deep Learning > Architectures > Transformers
Deep Learning > Techniques > Model Architecture
Deep Learning > Techniques > Pretraining
Machine Learning > Learning Types > Transfer Learning