2022
JMLR
JMLR 2022
Convergence Rates for Gaussian Mixtures of Experts
Abstract
We provide a theoretical treatment of over-specified Gaussian mixtures of experts with covariate-free gating networks. We establish the convergence rates of the maximum likelihood estimation (MLE) for these models. Our proof technique is based on a novel notion of algebraic independence of the expert functions. Drawing on optimal transport, we establish a connection between the algebraic independence of the expert functions and a certain class of partial differential equations (PDEs) with respect to the parameters. Exploiting this connection allows us to derive convergence rates for parameter estimation. [abs] [ pdf ][ bib ] © JMLR 2022. (edit, beta)
🌉
Interdisciplinary Bridge
— Machine Learning and Mathematics & Optimization
🐣
Hot Topic Early Bird
— mixture of expert
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
🧭
Keyword Pioneer
— gaussian mixture of expert
Authors
Topics
Machine Learning > Core Methods > Regression
Machine Learning > Optimization & Theory > Statistical Learning
Mathematics & Optimization > Mathematics > Statistics
Machine Learning > Core Methods > Probabilistic Modeling
Machine Learning > Optimization & Theory > Statistics
Mathematics & Optimization > Probability > Stochastic Processes