Theoretical guarantees for lifted samplers

TL;DR

This paper provides theoretical guarantees showing that lifted samplers, a class of MCMC methods, cannot have their asymptotic variances increase by more than a factor of two, regardless of the target distribution or algorithm specifics.

Contribution

It offers a general theoretical analysis demonstrating an upper bound on the variance increase for lifted samplers across various algorithms and target distributions.

Findings

Asymptotic variances cannot increase by more than a factor of 2 with lifted samplers.

The result applies broadly to different types of lifted samplers derived from various algorithms.

Lifted samplers have limited potential for variance increase, indicating they are generally safe to use.

Abstract

Lifted samplers form a class of Markov chain Monte Carlo methods which has drawn a lot attention in recent years due to superior performance in challenging Bayesian applications. A canonical example of lifted samplers is the one that is derived from a random walk Metropolis algorithm for a totally-ordered state space such as the integers or the real numbers. The lifted sampler is derived by splitting into two the proposal distribution: one part in the increasing direction, and the other part in the decreasing direction. It keeps following a direction, until a rejection occurs, upon which it flips the direction. In terms of asymptotic variances, it outperforms the random walk Metropolis algorithm, regardless of the target distribution, at no additional computational cost. Other studies show, however, that beyond this simple case, lifted samplers do not always outperform their Metropolis counterparts. In this paper, we leverage the celebrated work of Tierney (1998) to provide an analysis in a general framework encompassing a broad class of lifted samplers. Our finding is that, essentially, the asymptotic variances cannot increase by a factor of more than 2, regardless of the target distribution, the way the directions are induced, and the type of algorithm from which the lifted sampler is derived (be it a Metropolis--Hastings algorithm, a reversible jump algorithm, etc.). This result indicates that, while there is potentially a lot to gain from lifting a sampler, there is not much to lose.