← Back to papers
2006 JMLR JMLR 2006

Lower Bounds and Aggregation in Density Estimation

Abstract

In this paper we prove the optimality of an aggregation procedure. We prove lower bounds for aggregation of model selection type of M density estimators for the Kullback-Leibler divergence (KL), the Hellinger's distance and the L1-distance. The lower bound, with respect to the KL distance, can be achieved by the on-line type estimate suggested, among others, by Yang (2000a). Combining these results, we state that log M/n is an optimal rate of aggregation in the sense of Tsybakov (2003), where n is the sample size. [abs] [ pdf ][ bib ] © JMLR 2006. (edit, beta)

🌉 Interdisciplinary Bridge — Machine Learning and Mathematics & Optimization
📈 Trend Setter — Statistics
🧭 Keyword Pioneer — model aggregation
🐣 Hot Topic Early Bird — model selection
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Guillaume Lecue

Topics

Machine Learning > Core Methods > Representation Learning Machine Learning > Optimization & Theory > Statistical Learning Machine Learning > Optimization & Theory > Theory Mathematics & Optimization > Statistics

Keywords

model selection density estimation model aggregation kullback-leibler divergence lower bound optimal rate hellinger distance
Download PDF

Related papers

Superior Guarantees for Sequential Prediction and Lossless Compression via Alphabet Decomposition 2006
Action Elimination and Stopping Conditions for the Multi-Armed Bandit and Reinforcement Learning Problems 2006
Ensemble Pruning Via Semi-definite Programming 2006
Exact 1-Norm Support Vector Machines Via Unconstrained Convex Differentiable Minimization 2006
Kernel-Based Learning of Hierarchical Multilabel Classification Models 2006