Techniques for proving Asynchronous Convergence results for Markov Chain Monte Carlo methods

TL;DR

This paper explores the challenges and solutions for proving convergence of asynchronous parallel MCMC algorithms, extending recent theoretical insights to a broad class of methods beyond Gibbs sampling.

Contribution

It generalizes existing convergence theory for asynchronous MCMC, enabling understanding and ensuring convergence of various algorithms in parallel and distributed settings.

Findings

Recent theory explains divergence issues in asynchronous MCMC

A generic approach to analyze asynchronous MCMC algorithms

Guidelines to modify algorithms for guaranteed convergence

Abstract

Markov Chain Monte Carlo (MCMC) methods such as Gibbs sampling are finding widespread use in applied statistics and machine learning. These often lead to difficult computational problems, which are increasingly being solved on parallel and distributed systems such as compute clusters. Recent work has proposed running iterative algorithms such as gradient descent and MCMC in parallel asynchronously for increased performance, with good empirical results in certain problems. Unfortunately, for MCMC this parallelization technique requires new convergence theory, as it has been explicitly demonstrated to lead to divergence on some examples. Recent theory on Asynchronous Gibbs sampling describes why these algorithms can fail, and provides a way to alter them to make them converge. In this article, we describe how to apply this theory in a generic setting, to understand the asynchronous behavior of any MCMC algorithm, including those implemented using parameter servers, and those not based on Gibbs sampling.