Simulated Tempering and Swapping on Mean-Field Models

TL;DR

This paper introduces entropy dampening distributions for simulated tempering, demonstrating polynomial mixing times for certain models and highlighting the inefficiency of traditional tempering in some cases.

Contribution

It proposes entropy dampening as a new tempering distribution, improving convergence rates for specific mean-field models compared to traditional methods.

Findings

Entropy dampening achieves polynomial mixing times for asymmetric exponential and Ising models.

Traditional tempering converges slowly for the mean-field 3-state Potts model.

Slow mixing in traditional tempering can lead to exponentially longer mixing times than fixed-temperature methods.

Abstract

Simulated and parallel tempering are families of Markov Chain Monte Carlo algorithms where a temperature parameter is varied during the simulation to overcome bottlenecks to convergence due to multimodality. In this work we introduce and analyze the convergence for a set of new tempering distributions which we call \textit{entropy dampening}. For asymmetric exponential distributions and the mean field Ising model with and external field simulated tempering is known to converge slowly. We show that tempering with entropy dampening distributions mixes in polynomial time for these models. Examining slow mixing times of tempering more closely, we show that for the mean-field 3-state ferromagnetic Potts model, tempering converges slowly regardless of the temperature schedule chosen. On the other hand, tempering with entropy dampening distributions converges in polynomial time to stationarity. Finally we show that the slow mixing can be very expensive practically. In particular, the mixing time of simulated tempering is an exponential factor longer than the mixing time at the fixed temperature.