2020
EMNLP
EMNLP 2020
Dynamic Data Selection for Curriculum Learning via Ability Estimation
Abstract
AbstractCurriculum learning methods typically rely on heuristics to estimate the difficulty of training examples or the ability of the model. In this work, we propose replacing difficulty heuristics with learned difficulty parameters. We also propose Dynamic Data selection for Curriculum Learning via Ability Estimation (DDaCLAE), a strategy that probes model ability at each training epoch to select the best training examples at that point. We show that models using learned difficulty and/or ability outperform heuristic-based curriculum learning models on the GLUE classification tasks.
🌉
Interdisciplinary Bridge
— Deep Learning and Machine Learning and Natural Language Processing
🧭
Keyword Pioneer
— ability estimation
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
Authors
Topics
Machine Learning > Learning Types > Active Learning
Machine Learning > Learning Types > Continual Learning
Machine Learning > Optimization & Theory > Optimization
Natural Language Processing > Applications > Text Classification
Machine Learning > Learning Types > Multi-Task Learning
Deep Learning > Optimization & Theory > Neural Network Optimization
Machine Learning > Learning Paradigms > Curriculum Learning