← Back to papers
2012 NIPS NeurIPS 2012

Meta-Gaussian Information Bottleneck

Abstract

We present a reformulation of the information bottleneck (IB) problem in terms of copula, using the equivalence between mutual information and negative copula entropy. Focusing on the Gaussian copula we extend the analytical IB solution available for the multivariate Gaussian case to distributions with a Gaussian dependence structure but arbitrary marginal densities, also called meta-Gaussian distributions. This opens new possibles applications of IB to continuous data and provides a solution more robust to outliers.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Machine Learning and Mathematics & Optimization
🧭 Keyword Pioneer — gaussian copula
🐝 Cross-Pollinator — Artificial Intelligence, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Speech & Audio
🐣 Hot Topic Early Bird — probabilistic modeling

Authors

Melanie Rey , Volker Roth

Topics

Artificial Intelligence > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Core Methods > Representation Learning Machine Learning > Optimization & Theory > Statistical Learning Deep Learning > Models > Variational Inference Mathematics & Optimization > Mathematics > Information Theory Machine Learning > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Optimization & Theory > Information Theory Machine Learning > Optimization & Theory > Statistics

Keywords

probabilistic modeling variational inference information bottleneck mutual information outlier robustness gaussian copula continuous data
Download PDF

Related papers

Kernel Hyperalignment 2012
Fused sparsity and robust estimation for linear models with unknown variance 2012
Slice sampling normalized kernel-weighted completely random measure mixture models 2012
Scaling MPE Inference for Constrained Continuous Markov Random Fields with Consensus Optimization 2012
Matrix reconstruction with the local max norm 2012