← Back to papers
2012 NIPS NeurIPS 2012

Pointwise Tracking the Optimal Regression Function

Abstract

This paper examines the possibility of a `reject option' in the context of least squares regression. It is shown that using rejection it is theoretically possible to learn `selective' regressors that can $\epsilon$-pointwise track the best regressor in hindsight from the same hypothesis class, while rejecting only a bounded portion of the domain. Moreover, the rejected volume vanishes with the training set size, under certain conditions. We then develop efficient and exact implementation of these selective regressors for the case of linear regression. Empirical evaluation over a suite of real-world datasets corroborates the theoretical analysis and indicates that our selective regressors can provide substantial advantage by reducing estimation error.

🧭 Keyword Pioneer — selective regression
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Data Science & Analytics, Deep Learning, Interdisciplinary, Machine Learning, Mathematics & Optimization, Reinforcement Learning
📈 Trend Setter — Regression
🐣 Hot Topic Early Bird — learning theory

Authors

Yair Wiener , Ran El-Yaniv

Topics

Machine Learning > Core Methods > Classification Machine Learning > Core Methods > Regression Machine Learning > Learning Types > Weakly Supervised Learning Machine Learning > Optimization & Theory > Learning Theory Machine Learning > Learning Types > Supervised Learning Machine Learning > Learning Types > Regression

Keywords

learning theory reject option least squares regression selective regression pointwise tracking hypothesis class pointwise error estimation error
Download PDF

Related papers

Kernel Hyperalignment 2012
Fused sparsity and robust estimation for linear models with unknown variance 2012
Slice sampling normalized kernel-weighted completely random measure mixture models 2012
Scaling MPE Inference for Constrained Continuous Markov Random Fields with Consensus Optimization 2012
Matrix reconstruction with the local max norm 2012