← Back to papers
2006 NIPS NeurIPS 2006

PAC-Bayes Bounds for the Risk of the Majority Vote and the Variance of the Gibbs Classifier

Abstract

We propose new PAC-Bayes bounds for the risk of the weighted majority vote that depend on the mean and variance of the error of its associated Gibbs classifier. We show that these bounds can be smaller than the risk of the Gibbs classifier and can be arbitrarily close to zero even if the risk of the Gibbs classifier is close to 1/2. Moreover, we show that these bounds can be uniformly estimated on the training data for all possible posteriors Q. Moreover, they can be improved by using a large sample of unlabelled data.

🚀 Conference Pioneer — NIPS 2006
🌉 Interdisciplinary Bridge — Artificial Intelligence and Machine Learning
🧭 Keyword Pioneer — pac-bayes theory
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Data Science & Analytics, Deep Learning, Interdisciplinary, Machine Learning, Mathematics & Optimization
🐣 Hot Topic Early Bird — learning theory

Authors

Alexandre Lacasse , François Laviolette , Mario Marchand , Pascal Germain , Nicolas Usunier

Topics

Artificial Intelligence > Bayesian & Probabilistic > Bayesian Learning Machine Learning > Optimization & Theory > Learning Theory Machine Learning > Optimization & Theory > Statistical Learning Machine Learning > Bayesian & Probabilistic > Bayesian Learning Machine Learning > Bayesian & Probabilistic > Bayesian Inference

Keywords

learning theory ensemble learning pac-bayes theory majority vote gibbs classifier bound analysis variance estimation pac-bayes bound risk bound variance bound
Download PDF

Related papers

Temporal Coding using the Response Properties of Spiking Neurons 2006
Parameter Expanded Variational Bayesian Methods 2006
Effects of Stress and Genotype on Meta-parameter Dynamics in Reinforcement Learning 2006
Ordinal Regression by Extended Binary Classification 2006
Blind source separation for over-determined delayed mixtures 2006