← Back to papers
2013 NIPS NeurIPS 2013

Scalable Inference for Logistic-Normal Topic Models

Abstract

Logistic-normal topic models can effectively discover correlation structures among latent topics. However, their inference remains a challenge because of the non-conjugacy between the logistic-normal prior and multinomial topic mixing proportions. Existing algorithms either make restricting mean-field assumptions or are not scalable to large-scale applications. This paper presents a partially collapsed Gibbs sampling algorithm that approaches the provably correct distribution by exploring the ideas of data augmentation. To improve time efficiency, we further present a parallel implementation that can deal with large-scale applications and learn the correlation structures of thousands of topics from millions of documents. Extensive empirical results demonstrate the promise.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Machine Learning and Natural Language Processing
📈 Trend Setter — Large Language Models
🧭 Keyword Pioneer — latent topics
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio
🐣 Hot Topic Early Bird — data augmentation

Authors

Jianfei Chen , Jun Zhu , Zi Wang , Xun Zheng , Bo Zhang

Topics

Artificial Intelligence > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Learning Types > Unsupervised Learning Machine Learning > Optimization & Theory > Bayesian Inference Natural Language Processing > Resources & Methods > Large Language Models Machine Learning > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Bayesian & Probabilistic > Bayesian Inference Natural Language Processing > Applications > Topic Modeling

Keywords

bayesian inference probabilistic inference data augmentation parallel computing gibbs sampling scalable inference latent topics logistic-normal models latent topic correlation topic model latent topic logistic-normal topic model topic correlation
Download PDF

Related papers

Latent Structured Active Learning 2013
On Flat versus Hierarchical Classification in Large-Scale Taxonomies 2013
Generalized Method-of-Moments for Rank Aggregation 2013
Third-Order Edge Statistics: Contour Continuation, Curvature, and Cortical Connections 2013
Accelerated Mini-Batch Stochastic Dual Coordinate Ascent 2013