← Back to papers
2013 NIPS NeurIPS 2013

Integrated Non-Factorized Variational Inference

Abstract

We present a non-factorized variational method for full posterior inference in Bayesian hierarchical models, with the goal of capturing the posterior variable dependencies via efficient and possibly parallel computation. Our approach unifies the integrated nested Laplace approximation (INLA) under the variational framework. The proposed method is applicable in more challenging scenarios than typically assumed by INLA, such as Bayesian Lasso, which is characterized by the non-differentiability of the $\ell_{1}$ norm arising from independent Laplace priors. We derive an upper bound for the Kullback-Leibler divergence, which yields a fast closed-form solution via decoupled optimization. Our method is a reliable analytic alternative to Markov chain Monte Carlo (MCMC), and it results in a tighter evidence lower bound than that of mean-field variational Bayes (VB) method.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Deep Learning and Machine Learning
🧭 Keyword Pioneer — inla
🐝 Cross-Pollinator — Artificial Intelligence, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Speech & Audio
📈 Trend Setter — Variational Inference
🐣 Hot Topic Early Bird — kullback-leibler divergence

Authors

Shaobo Han , Xuejun Liao , Lawrence Carin

Topics

Artificial Intelligence > Bayesian & Probabilistic > Bayesian Learning Artificial Intelligence > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Optimization & Theory > Bayesian Inference Machine Learning > Optimization & Theory > Optimization Deep Learning > Models > Variational Inference Artificial Intelligence > Bayesian & Probabilistic > Bayesian Inference Machine Learning > Bayesian & Probabilistic > Bayesian Inference Machine Learning > Bayesian & Probabilistic > Variational Inference Artificial Intelligence > Bayesian & Probabilistic > Variational Inference

Keywords

variational inference bayesian inference markov chain monte carlo laplace approximation kullback-leibler divergence posterior inference hierarchical model inla full posterior inference mean-field variational bayes integrated nested laplace approximation evidence lower bound bayesian hierarchical model non-factorized variational
Download PDF

Related papers

Latent Structured Active Learning 2013
On Flat versus Hierarchical Classification in Large-Scale Taxonomies 2013
Generalized Method-of-Moments for Rank Aggregation 2013
Third-Order Edge Statistics: Contour Continuation, Curvature, and Cortical Connections 2013
Accelerated Mini-Batch Stochastic Dual Coordinate Ascent 2013