Ensuring Rapid Mixing and Low Bias for Asynchronous Gibbs Sampling

TL;DR

This paper investigates the theoretical properties of asynchronous Gibbs sampling, focusing on bias and mixing time, and provides insights that align with empirical observations to improve parallel MCMC methods.

Contribution

The paper offers a novel theoretical analysis of asynchronous Gibbs sampling, addressing bias and mixing time, which were previously poorly understood.

Findings

Theoretical bounds on bias in asynchronous Gibbs sampling

Analysis of mixing time under asynchronous execution

Experimental validation of theoretical results

Abstract

Gibbs sampling is a Markov chain Monte Carlo technique commonly used for estimating marginal distributions. To speed up Gibbs sampling, there has recently been interest in parallelizing it by executing asynchronously. While empirical results suggest that many models can be efficiently sampled asynchronously, traditional Markov chain analysis does not apply to the asynchronous case, and thus asynchronous Gibbs sampling is poorly understood. In this paper, we derive a better understanding of the two main challenges of asynchronous Gibbs: bias and mixing time. We show experimentally that our theoretical results match practical outcomes.