← Back to papers
2013 NIPS NeurIPS 2013

RNADE: The real-valued neural autoregressive density-estimator

Abstract

We introduce RNADE, a new model for joint density estimation of real-valued vectors. Our model calculates the density of a datapoint as the product of one-dimensional conditionals modeled using mixture density networks with shared parameters. RNADE learns a distributed representation of the data, while having a tractable expression for the calculation of densities. A tractable likelihood allows direct comparison with other methods and training by standard gradient-based optimizers. We compare the performance of RNADE on several datasets of heterogeneous and perceptual data, finding it outperforms mixture models in all but one case.

🌉 Interdisciplinary Bridge — Deep Learning and Machine Learning
🧭 Keyword Pioneer — neural autoregressive density
🐣 Hot Topic Early Bird — generative model
🐝 Cross-Pollinator — Artificial Intelligence, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Speech & Audio

Authors

Benigno Uria , Iain Murray , Hugo Larochelle

Topics

Machine Learning > Core Methods > Classification Machine Learning > Core Methods > Regression Machine Learning > Learning Types > Unsupervised Learning Deep Learning > Architectures > Neural Networks Deep Learning > Models > Generative Models Deep Learning > Learning Types > Representation Learning

Keywords

unsupervised learning density estimation neural autoregressive model neural autoregressive density joint density estimation real-valued data tractable likelihood generative model mixture model autoregressive model mixture density network neural network neural autoregressive density estimator real-valued datum
Download PDF

Related papers

Latent Structured Active Learning 2013
On Flat versus Hierarchical Classification in Large-Scale Taxonomies 2013
Generalized Method-of-Moments for Rank Aggregation 2013
Third-Order Edge Statistics: Contour Continuation, Curvature, and Cortical Connections 2013
Accelerated Mini-Batch Stochastic Dual Coordinate Ascent 2013