← Back to papers
2011 NIPS NeurIPS 2011

Learning to Learn with Compound HD Models

Abstract

We introduce HD (or ``Hierarchical-Deep'') models, a new compositional learning architecture that integrates deep learning models with structured hierarchical Bayesian models. Specifically we show how we can learn a hierarchical Dirichlet process (HDP) prior over the activities of the top-level features in a Deep Boltzmann Machine (DBM). This compound HDP-DBM model learns to learn novel concepts from very few training examples, by learning low-level generic features, high-level features that capture correlations among low-level features, and a category hierarchy for sharing priors over the high-level features that are typical of different kinds of concepts. We present efficient learning and inference algorithms for the HDP-DBM model and show that it is able to learn new concepts from very few examples on CIFAR-100 object recognition, handwritten character recognition, and human motion capture datasets.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Deep Learning and Machine Learning
📈 Trend Setter — Meta-Learning
🧭 Keyword Pioneer — few-shot learning
🐣 Hot Topic Early Bird — few-shot learning
🐝 Cross-Pollinator — Artificial Intelligence, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Speech & Audio
🌱 Topic Pioneer — Meta-Learning

Authors

Antonio Torralba , Joshua B. Tenenbaum , Ruslan Salakhutdinov

Topics

Artificial Intelligence > Bayesian & Probabilistic > Bayesian Learning Artificial Intelligence > Learning Paradigms > Meta-Learning Deep Learning > Architectures > Neural Networks Deep Learning > Models > Generative Models Machine Learning > Bayesian & Probabilistic > Bayesian Learning Machine Learning > Learning Paradigms > Few-Shot Learning Machine Learning > Learning Paradigms > Meta-Learning Deep Learning > Learning Types > Meta-Learning

Keywords

few-shot learning hierarchical dirichlet processes compositional learning bayesian model hierarchical dirichlet process concept learning hierarchical-deep models hierarchical bayesian model deep boltzmann machine
Download PDF

Related papers

Co-Training for Domain Adaptation 2011
The Local Rademacher Complexity of Lp-Norm Multiple Kernel Learning 2011
Learning to Agglomerate Superpixel Hierarchies 2011
A Reinforcement Learning Theory for Homeostatic Regulation 2011
A Global Structural EM Algorithm for a Model of Cancer Progression 2011