Lifting -- A nonreversible Markov chain Monte Carlo Algorithm

TL;DR

This paper reviews nonreversible Markov chain Monte Carlo algorithms, especially lifted chains, demonstrating their potential for faster convergence in sampling tasks compared to traditional reversible methods.

Contribution

It introduces the concept of lifted nonreversible Markov chains, discusses their construction, and illustrates their efficiency through various sampling examples.

Findings

Nonreversible chains can outperform reversible ones with up to a square root speedup.

Lifted Markov chains are constructed by enlarging the state space and modifying transition rates.

Examples include sampling on rings, tori, and Ising models demonstrating improved efficiency.

Abstract

Markov chain Monte Carlo algorithms are invaluable tools for exploring stationary properties of physical systems, especially in situations where direct sampling is unfeasible. Common implementations of Monte Carlo algorithms employ reversible Markov chains. Reversible chains obey detailed balance and thus ensure that the system will eventually relax to equilibrium. Detailed balance is not necessary for convergence to equilibrium. We review nonreversible Markov chains, which violate detailed balance, and yet still relax to a given target stationary distribution. In particular cases, nonreversible Markov chains are substantially better at sampling than the conventional reversible Markov chains with up to a square root improvement in the convergence time to the steady state. One kind of nonreversible Markov chain is constructed from the reversible ones by enlarging the state space and by modifying and adding extra transition rates to create non-reversible moves. Because of the augmentation of the state space, such chains are often referred to as lifted Markov Chains. We illustrate the use of lifted Markov chains for efficient sampling for several examples. The examples include sampling on a ring, sampling on a torus, the Ising model on a complete graph, and the one-dimensional Ising model. We also provide a pseudocode implementation, review related work, and discuss the applicability of such methods.