← Back to papers
2014 ICML ICML 2014

Sample-based approximate regularization

Abstract

We introduce a method for regularizing linearly parameterized functions using general derivative-based penalties, which relies on sampling as well as finite-difference approximations of the relevant derivatives. We call this approach sample-based approximate regularization (SAR). We provide theoretical guarantees on the fidelity of such regularizers, compared to those they approximate, and prove that the approximations converge efficiently. We also examine the empirical performance of SAR on several datasets.

📈 Trend Setter — Loss Functions
🧭 Keyword Pioneer — derivative-based penalty
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Data Science & Analytics, Deep Learning, Machine Learning, Mathematics & Optimization, Reinforcement Learning
🐣 Hot Topic Early Bird — function approximation

Authors

Philip Bachman , Amir-massoud Farahmand , Doina Precup

Topics

Machine Learning > Core Methods > Regression Machine Learning > Optimization & Theory > Loss Functions Machine Learning > Optimization & Theory > Optimization Machine Learning > Learning Types > Supervised Learning

Keywords

function approximation derivative-based penalty finite-difference approximation linear parameterization
Download PDF

Related papers

Demystifying Information-Theoretic Clustering 2014
Margins, Kernels and Non-linear Smoothed Perceptrons 2014
Large-Margin Metric Learning for Constrained Partitioning Problems 2014
Efficient Approximation of Cross-Validation for Kernel Methods using Bouligand Influence Function 2014
Generalized Exponential Concentration Inequality for Renyi Divergence Estimation 2014