← Back to papers
2013 NIPS NeurIPS 2013

Dirty Statistical Models

Abstract

We provide a unified framework for the high-dimensional analysis of “superposition-structured” or “dirty” statistical models: where the model parameters are a “superposition” of structurally constrained parameters. We allow for any number and types of structures, and any statistical model. We consider the general class of $M$-estimators that minimize the sum of any loss function, and an instance of what we call a “hybrid” regularization, that is the infimal convolution of weighted regularization functions, one for each structural component. We provide corollaries showcasing our unified framework for varied statistical models such as linear regression, multiple regression and principal component analysis, over varied superposition structures.

🌉 Interdisciplinary Bridge — Machine Learning and Mathematics & Optimization
🧭 Keyword Pioneer — superposition structure
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy
📈 Trend Setter — Multi-Task Learning
🐣 Hot Topic Early Bird — statistical learning

Authors

Eunho Yang , Pradeep K Ravikumar

Topics

Machine Learning > Core Methods > Regression Machine Learning > Optimization & Theory > Optimization Machine Learning > Optimization & Theory > Statistical Learning Mathematics & Optimization > Optimization > Continuous Optimization Machine Learning > Core Methods > Feature Selection Machine Learning > Core Methods > Feature Learning Machine Learning > Core Methods > Multi-Task Learning Mathematics & Optimization > Optimization > Sparse Optimization Machine Learning > Core Methods > Optimization Machine Learning > Optimization & Theory > Sparse Optimization Machine Learning > Core Methods > Structured Prediction Machine Learning > Learning Types > Sparse Learning

Keywords

statistical learning regularization convex optimization feature selection sparse estimation multi-task feature learning high-dimensional analysis sparse optimization m-estimators superposition structure sparse model hybrid regularization
Download PDF

Related papers

Latent Structured Active Learning 2013
On Flat versus Hierarchical Classification in Large-Scale Taxonomies 2013
Generalized Method-of-Moments for Rank Aggregation 2013
Third-Order Edge Statistics: Contour Continuation, Curvature, and Cortical Connections 2013
Accelerated Mini-Batch Stochastic Dual Coordinate Ascent 2013