Parallelizing MCMC Sampling via Space Partitioning

TL;DR

This paper introduces a parallel MCMC algorithm that partitions the parameter space to enable independent sampling, reducing computation time, improving multimodal sampling, and providing integral estimates.

Contribution

The novel method allows parallelization of MCMC by space partitioning, enhancing efficiency and sampling quality for complex densities.

Findings

Reduces sampling wall-clock time through parallelization.

Improves sampling of multimodal densities.

Provides estimates of the target density integral.

Abstract

Efficient sampling of many-dimensional and multimodal density functions is a task of great interest in many research fields. We describe an algorithm that allows parallelizing inherently serial Markov chain Monte Carlo (MCMC) sampling by partitioning the space of the function parameters into multiple subspaces and sampling each of them independently. The samples of the different subspaces are then reweighted by their integral values and stitched back together. This approach allows reducing sampling wall-clock time by parallel operation. It also improves sampling of multimodal target densities and results in less correlated samples. Finally, the approach yields an estimate of the integral of the target density function.