Unbiased Markov chain Monte Carlo for intractable target distributions

TL;DR

This paper introduces a coupling-based method to produce unbiased MCMC estimators for intractable target distributions, overcoming burn-in bias and enabling finite-time, parallelizable inference in complex models.

Contribution

It extends coupling techniques to polynomially ergodic Markov chains, providing theoretically valid unbiased estimators for intractable distributions.

Findings

Unbiased estimators outperform standard MCMC in finite samples.

The method is effective for state space and Ising models.

Theoretical analysis confirms estimator validity.

Abstract

Performing numerical integration when the integrand itself cannot be evaluated point-wise is a challenging task that arises in statistical analysis, notably in Bayesian inference for models with intractable likelihood functions. Markov chain Monte Carlo (MCMC) algorithms have been proposed for this setting, such as the pseudo-marginal method for latent variable models and the exchange algorithm for a class of undirected graphical models. As with any MCMC algorithm, the resulting estimators are justified asymptotically in the limit of the number of iterations, but exhibit a bias for any fixed number of iterations due to the Markov chains starting outside of stationarity. This "burn-in" bias is known to complicate the use of parallel processors for MCMC computations. We show how to use coupling techniques to generate unbiased estimators in finite time, building on recent advances for generic MCMC algorithms. We establish the theoretical validity of some of these procedures by extending existing results to cover the case of polynomially ergodic Markov chains. The efficiency of the proposed estimators is compared with that of standard MCMC estimators, with theoretical arguments and numerical experiments including state space models and Ising models.