← Back to papers
2011 AISTATS AISTATS 2011

Deep Learners Benefit More from Out-of-Distribution Examples

Abstract

Recent theoretical and empirical work in statistical machine learning has demonstrated the potential of learning algorithms for deep architectures, i.e., function classes obtained by composing multiple levels of representation. The hypothesis evaluated here is that intermediate levels of representation, because they can be shared across tasks and examples from different but related distributions, can yield even more benefits. Comparative experiments were performed on a large-scale handwritten character recognition setting with 62 classes (upper case, lower case, digits), using both a multi-task setting and perturbed examples in order to obtain out-of-distribution examples. The results agree with the hypothesis, and show that a deep learner did beat previously published results and reached human-level performance.

🌉 Interdisciplinary Bridge — Deep Learning and Machine Learning
📈 Trend Setter — Transfer Learning
🧭 Keyword Pioneer — character recognition
🐣 Hot Topic Early Bird — representation learning
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Yoshua Bengio , Frédéric Bastien , Arnaud Bergeron , Nicolas Boulanger–Lewandowski , Thomas Breuel , Youssouf Chherawala , Moustapha Cisse , Myriam Côté , Dumitru Erhan , Jeremy Eustache , Xavier Glorot , Xavier Muller , Sylvain Pannetier Lebeuf , Razvan Pascanu , Salah Rifai , François Savard , Guillaume Sicard

Topics

Machine Learning > Core Methods > Representation Learning Machine Learning > Learning Types > Semi-Supervised Learning Deep Learning > Architectures > Neural Networks Machine Learning > Learning Paradigms > Transfer Learning Machine Learning > Learning Paradigms > Multi-Task Learning Deep Learning > Learning Types > Deep Learning

Keywords

representation learning multi-task learning deep learning character recognition handwritten character recognition out-of-distribution learning
Download PDF

Related papers

Tighter Relaxations for MAP-MRF Inference: A Local Primal-Dual Gap based Separation Algorithm 2011
Bagged Structure Learning of Bayesian Network 2011
Convergent Decomposition Solvers for Tree-reweighted Free Energies 2011
Follow-the-Regularized-Leader and Mirror Descent: Equivalence Theorems and L1 Regularization 2011
Dimensionality Reduction for Spectral Clustering 2011