← Back to papers
2012 NIPS NeurIPS 2012

Minimax Multi-Task Learning and a Generalized Loss-Compositional Paradigm for MTL

Abstract

Since its inception, the modus operandi of multi-task learning (MTL) has been to minimize the task-wise mean of the empirical risks. We introduce a generalized loss-compositional paradigm for MTL that includes a spectrum of formulations as a subfamily. One endpoint of this spectrum is minimax MTL: a new MTL formulation that minimizes the maximum of the tasks' empirical risks. Via a certain relaxation of minimax MTL, we obtain a continuum of MTL formulations spanning minimax MTL and classical MTL. The full paradigm itself is loss-compositional, operating on the vector of empirical risks. It incorporates minimax MTL, its relaxations, and many new MTL formulations as special cases. We show theoretically that minimax MTL tends to avoid worst case outcomes on newly drawn test tasks in the learning to learn (LTL) test setting. The results of several MTL formulations on synthetic and real problems in the MTL and LTL test settings are encouraging.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Machine Learning
📈 Trend Setter — Continual Learning
🧭 Keyword Pioneer — minimax optimization
🐣 Hot Topic Early Bird — multi-task learning
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Nishant Mehta , Dongryeol Lee , Alexander G. Gray

Topics

Artificial Intelligence > Learning Paradigms > Transfer Learning Machine Learning > Core Methods > Representation Learning Machine Learning > Learning Types > Continual Learning Machine Learning > Optimization & Theory > Optimization Machine Learning > Learning Types > Multi-Task Learning Machine Learning > Core Methods > Optimization Artificial Intelligence > Learning Paradigms > Multi-Task Learning Machine Learning > Learning Paradigms > Multi-Task Learning

Keywords

multi-task learning transfer learning task generalization minimax optimization empirical risk minimization worst-case optimization empirical risk task balancing learning to learn
Download PDF

Related papers

Kernel Hyperalignment 2012
Fused sparsity and robust estimation for linear models with unknown variance 2012
Slice sampling normalized kernel-weighted completely random measure mixture models 2012
Scaling MPE Inference for Constrained Continuous Markov Random Fields with Consensus Optimization 2012
Matrix reconstruction with the local max norm 2012