← Back to papers
2013 AISTATS AISTATS 2013

Distributed and Adaptive Darting Monte Carlo through Regenerations

Abstract

Darting Monte Carlo (DMC) is a MCMC procedure designed to effectively mix between multiple modes of a probability distribution. We propose an adaptive and distributed version of this method by using regenerations. This allows us to run multiple chains in parallel and adapt the shape of the jump regions as well as all other aspects of the Markov chain on the fly. We show that this significantly improves the performance of DMC because 1) a population of chains has a higher chance of finding the modes in the distribution, 2) jumping between modes becomes easier due to the adaptation of their shapes, 3) computation is much more efficient due to parallelization across multiple processors. While the curse of dimensionality is a challenge for both DMC and regeneration, we find that their combination ameliorates this issue slightly.

🌉 Interdisciplinary Bridge — Machine Learning and Mathematics & Optimization
🧭 Keyword Pioneer — mode jumping
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Sungjin Ahn , Yutian Chen , Max Welling

Topics

Machine Learning > Optimization & Theory > Bayesian Inference Machine Learning > Optimization & Theory > Distributed Learning Mathematics & Optimization > Optimization > Stochastic Methods Machine Learning > Optimization & Theory > Stochastic Methods Machine Learning > Bayesian & Probabilistic > Bayesian Inference

Keywords

markov chain monte carlo adaptive algorithm parallel computation parallel sampling multi-modal distribution mode jumping darting monte carlo
Download PDF

Related papers

Consensus Ranking with Signed Permutations 2013
Ultrahigh Dimensional Feature Screening via RKHS Embeddings 2013
Collapsed Variational Bayesian Inference for Hidden Markov Models 2013
Learning Social Infectivity in Sparse Low-rank Networks Using Multi-dimensional Hawkes Processes 2013
Evidence Estimation for Bayesian Partially Observed MRFs 2013