Diffusion Generative Modelling for Divide-and-Conquer MCMC

TL;DR

This paper introduces a diffusion generative modeling approach to improve the merging step in divide-and-conquer MCMC, enabling more efficient and assumption-free density approximation of subposteriors, especially in high dimensions.

Contribution

It presents a novel use of diffusion generative models for non-parametric density estimation in divide-and-conquer MCMC, outperforming existing methods in challenging scenarios.

Findings

Outperforms existing merging methods on complex problems

Scales more efficiently to high-dimensional data

Provides accurate density approximations without distributional assumptions

Abstract

Divide-and-conquer MCMC is a strategy for parallelising Markov Chain Monte Carlo sampling by running independent samplers on disjoint subsets of a dataset and merging their output. An ongoing challenge in the literature is to efficiently perform this merging without imposing distributional assumptions on the posteriors. We propose using diffusion generative modelling to fit density approximations to the subposterior distributions. This approach outperforms existing methods on challenging merging problems, while its computational cost scales more efficiently to high dimensional problems than existing density estimation approaches.