2011
NIPS
NeurIPS 2011
Non-conjugate Variational Message Passing for Multinomial and Binary Regression
Abstract
Variational Message Passing (VMP) is an algorithmic implementation of the Variational Bayes (VB) method which applies only in the special case of conjugate exponential family models. We propose an extension to VMP, which we refer to as Non-conjugate Variational Message Passing (NCVMP) which aims to alleviate this restriction while maintaining modularity, allowing choice in how expectations are calculated, and integrating into an existing message-passing framework: Infer.NET. We demonstrate NCVMP on logistic binary and multinomial regression. In the multinomial case we introduce a novel variational bound for the softmax factor which is tighter than other commonly used bounds whilst maintaining computational tractability.
🌉
Interdisciplinary Bridge
— Artificial Intelligence and Deep Learning and Machine Learning
🧭
Keyword Pioneer
— non-conjugate variational inference
🐣
Hot Topic Early Bird
— variational inference
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
Authors
Topics
Artificial Intelligence > Bayesian & Probabilistic > Bayesian Learning
Machine Learning > Core Methods > Regression
Machine Learning > Optimization & Theory > Bayesian Inference
Deep Learning > Models > Variational Inference
Machine Learning > Learning Types > Supervised Learning
Machine Learning > Bayesian & Probabilistic > Bayesian Inference
Machine Learning > Bayesian & Probabilistic > Variational Inference