← Back to papers
2007 NIPS NeurIPS 2007

One-Pass Boosting

Abstract

This paper studies boosting algorithms that make a single pass over a set of base classi(cid:2)ers. We (cid:2)rst analyze a one-pass algorithm in the setting of boosting with diverse base classi(cid:2)ers. Our guarantee is the same as the best proved for any boosting algo- rithm, but our one-pass algorithm is much faster than previous approaches. We next exhibit a random source of examples for which a (cid:147)picky(cid:148) variant of Ad- aBoost that skips poor base classi(cid:2)ers can outperform the standard AdaBoost al- gorithm, which uses every base classi(cid:2)er, by an exponential factor. Experiments with Reuters and synthetic data show that one-pass boosting can sub- stantially improve on the accuracy of Naive Bayes, and that picky boosting can sometimes lead to a further improvement in accuracy.

📈 Trend Setter — Neural Network Optimization
🐝 Cross-Pollinator — Artificial Intelligence, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Speech & Audio
🌱 Topic Pioneer — Ensemble Learning
🧭 Keyword Pioneer — base classifier
🐣 Hot Topic Early Bird — ensemble learning

Authors

Zafer Barutcuoglu , Phil Long , Rocco Servedio

Topics

Machine Learning > Core Methods > Classification Machine Learning > Optimization & Theory > Neural Network Optimization Machine Learning > Learning Types > Supervised Learning Machine Learning > Learning Types > Ensemble Learning Machine Learning > Core Methods > Ensemble Methods Machine Learning > Learning Types > Classification Machine Learning > Learning Types > Ensemble Methods Machine Learning > Core Methods > Ensemble Learning

Keywords

ensemble learning classification adaboost boosting algorithm base classifier
Download PDF

Related papers

Exponential Family Predictive Representations of State 2007
Privacy-Preserving Belief Propagation and Sampling 2007
Efficient Principled Learning of Thin Junction Trees 2007
How SVMs can estimate quantiles and the median 2007
Rapid Inference on a Novel AND/OR graph for Object Detection, Segmentation and Parsing 2007