← Back to papers
2021 NIPS NeurIPS 2021

Boosting with Multiple Sources

Abstract

We study the problem of learning accurate ensemble predictors, in particular boosting, in the presence of multiple source domains. We show that the standard convex combination ensembles in general cannot succeed in this scenario and adopt instead a domain-weighted combination. We introduce and analyze a new boosting algorithm, MULTIBOOST, for this scenario and show that it benefits from favorable theoretical guarantees. We also report the results of several experiments with our algorithm demonstrating that it outperforms natural baselines on multi-source text-based, image-based and tabular data. We further present an extension of our algorithm to the federated learning scenario and report favorable experimental results for that setting as well. Additionally, we describe in detail an extension of our algorithm to the multi-class setting, MCMULTIBOOST, for which we also report experimental results.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Machine Learning
🧭 Keyword Pioneer — multiple source domain
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Corinna Cortes , Mehryar Mohri , Dmitry Storcheus , Ananda Theertha Suresh

Topics

Artificial Intelligence > Learning Paradigms > Federated Learning Machine Learning > Core Methods > Classification Machine Learning > Optimization & Theory > Optimization Machine Learning > Learning Paradigms > Federated Learning Machine Learning > Learning Types > Ensemble Learning Machine Learning > Learning Paradigms > Multi-Task Learning

Keywords

federated learning multi-source learning ensemble learning ensemble predictor multiple source domain domain-weighted combination
Download PDF

Related papers

Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data 2021
On Model Calibration for Long-Tailed Object Detection and Instance Segmentation 2021
Test-Time Personalization with a Transformer for Human Pose Estimation 2021
NTopo: Mesh-free Topology Optimization using Implicit Neural Representations 2021
Scalable Intervention Target Estimation in Linear Models 2021