Uniform Ergodicity of Parallel Tempering With Efficient Local Exploration

TL;DR

This paper proves that non-reversible parallel tempering (NRPT) algorithms achieve uniform geometric ergodicity under efficient local exploration, with convergence rates linked to the global communication barrier, outperforming reversible methods.

Contribution

The paper establishes the uniform ergodicity of NRPT under a model of efficient local exploration and compares it to reversible parallel tempering, showing NRPT's superior convergence properties.

Findings

NRPT is uniformly ergodic under certain conditions.

Convergence rates are inversely related to the global communication barrier.

Results align with empirical observations for complex distributions.

Abstract

Non-reversible parallel tempering (NRPT) is an effective algorithm for sampling from target distributions with complex geometry, such as those arising from posterior distributions of weakly identifiable and high-dimensional Bayesian models. In this work we establish the uniform (geometric) ergodicity of NRPT under a model of efficient local exploration. The uniform ergodicity log rates are inversely proportional to an easily-estimable divergence, the global communication barrier (GCB), which was recently introduced in the literature. We obtain analogous ergodicity results for classical reversible parallel tempering, providing new evidence that NRPT dominates its reversible counterpart. Our results are based on an analysis of the hitting time of a continuous-time persistent random walk, which is also of independent interest. The rates that we obtain reflect real experiments well for distributions where global exploration is not possible without parallel tempering.