← Back to papers
2009 NIPS NeurIPS 2009

Asymptotically Optimal Regularization in Smooth Parametric Models

Abstract

Many types of regularization schemes have been employed in statistical learning, each one motivated by some assumption about the problem domain. In this paper, we present a unified asymptotic analysis of smooth regularizers, which allows us to see how the validity of these assumptions impacts the success of a particular regularizer. In addition, our analysis motivates an algorithm for optimizing regularization parameters, which in turn can be analyzed within our framework. We apply our analysis to several examples, including hybrid generative-discriminative learning and multi-task learning.

🧭 Keyword Pioneer — smooth regularizers
🐣 Hot Topic Early Bird — multi-task learning
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
📈 Trend Setter — Multi-Task Learning

Authors

Percy Liang , Guillaume Bouchard , Francis R. Bach , Michael I. Jordan

Topics

Machine Learning > Core Methods > Regression Machine Learning > Optimization & Theory > Learning Theory Machine Learning > Optimization & Theory > Optimization Machine Learning > Optimization & Theory > Statistical Learning Machine Learning > Optimization & Theory > Theory Machine Learning > Core Methods > Multi-Task Learning Machine Learning > Optimization & Theory > Regularization

Keywords

statistical learning learning theory multi-task learning regularization asymptotic analysis parameter optimization smooth regularizers parametric model generative-discriminative learning
Download PDF

Related papers

Solving Stochastic Games 2009
Bilinear classifiers for visual recognition 2009
Zero-shot Learning with Semantic Output Codes 2009
Matrix Completion from Power-Law Distributed Samples 2009
Heavy-Tailed Symmetric Stochastic Neighbor Embedding 2009