The Coupled Rejection Sampler

TL;DR

This paper introduces a coupled rejection-sampling method that efficiently generates coupled samples from arbitrary distributions with bounded variance in execution time, applicable to Gaussian couplings, rare event sampling, particle filtering, and unbiased MCMC.

Contribution

The paper presents a novel coupled rejection-sampling algorithm with finite variance properties and optimized coupling for multivariate Gaussians, enhancing sampling efficiency in various probabilistic methods.

Findings

Variance of execution time remains finite as marginals approach each other.

Derived positive lower bounds for Gaussian coupling probabilities.

Demonstrated applications in particle filtering and unbiased MCMC.

Abstract

We propose a coupled rejection-sampling method for sampling from couplings of arbitrary distributions. The method relies on accepting or rejecting coupled samples coming from dominating marginals. Contrary to existing acceptance-rejection coupling methods, the variance of the execution time of the proposed method is limited and stays finite as the two target marginals approach each other in the sense of the total variation norm. In the important special case of coupling multivariate Gaussians with different means and covariances, we derive positive lower bounds for the resulting coupling probability of our algorithm, and we then show how the coupling method can be optimized in closed form. Finally, we show how we can modify the coupled rejection-sampling method to propose from coupled ensemble of proposals, so as to asymptotically recover a maximal coupling. We then apply the method to the problem of coupling rare events samplers, derive a parallel coupled resampling algorithm to use in particle filtering, and show how the coupled rejection-sampler can be used to speed up unbiased MCMC methods based on couplings.