Optimal Temperature Spacing for Regionally Weight-preserving Tempering

TL;DR

This paper investigates optimal temperature spacings for regionally weight-preserving parallel tempering in high-dimensional MCMC, proposing that spacings should scale as the inverse square root of the dimension to optimize swap acceptance rates.

Contribution

It derives a theoretical guideline for choosing temperature spacings in weight-preserving tempering, enhancing the efficiency of sampling multimodal distributions.

Findings

Temperature spacings should scale as O(d^{-1/2})

Optimal swap acceptance rate is between 0 and 0.234

Guidelines improve exploration of multimodal targets

Abstract

Parallel tempering is popular method for allowing MCMC algorithms to properly explore a $d$-dimensional multimodal target density. One problem with traditional power-based parallel tempering for multimodal targets is that the proportion of probability mass associated to the modes can change for different inverse-temperature values, sometimes dramatically so. Complementary work by the authors proposes a novel solution involving auxiliary targets that preserve regional weight upon powering up the density. This paper attempts to address the question of how to choose the temperature spacings in an optimal way when using this type of weight-preserving approach.The problem is analysed in a tractable setting for computation of the expected squared jumping distance which can then be optimised with regards to a tuning parameter. The conclusion is that for an appropriately constructed regionally weight-preserved tempering algorithm targeting a $d$-dimensional target distribution, the consecutive temperature spacings should behave as $\mathcal{O}\left(d^{-1/2}\right)$ and this induces an optimal acceptance rate for temperature swap moves that lies in the interval $[0,0.234]$.