2010
NIPS
NeurIPS 2010
Online Learning for Latent Dirichlet Allocation
Abstract
We develop an online variational Bayes (VB) algorithm for Latent Dirichlet Allocation (LDA). Online LDA is based on online stochastic optimization with a natural gradient step, which we show converges to a local optimum of the VB objective function. It can handily analyze massive document collections, including those arriving in a stream. We study the performance of online LDA in several ways, including by fitting a 100-topic topic model to 3.3M articles from Wikipedia in a single pass. We demonstrate that online LDA finds topic models as good or better than those found with batch VB, and in a fraction of the time.
🌱
Topic Pioneer
— Large Language Models
📈
Trend Setter
— Language Modeling
🧭
Keyword Pioneer
— document analysis
🐣
Hot Topic Early Bird
— online learning
🐝
Cross-Pollinator
— Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
🌉
Interdisciplinary Bridge
— Machine Learning and Natural Language Processing
Authors
Topics
Natural Language Processing > Generation > Language Modeling
Natural Language Processing > Resources & Methods > Large Language Models
Machine Learning > Bayesian & Probabilistic > Probabilistic Modeling
Machine Learning > Core Methods > Topic Modeling
Machine Learning > Learning Paradigms > Online Learning