Control of probability flow in Markov chain Monte Carlo -- Nonreversibility and lifting

TL;DR

This paper reviews how breaking detailed balance and introducing probability flows in Markov chain Monte Carlo methods, especially through lifting techniques, can significantly improve sampling efficiency in complex systems.

Contribution

It provides a comprehensive review of practical approaches to implementing nonreversible MCMC methods, highlighting the effectiveness of lifting techniques.

Findings

Nonreversible MCMC methods outperform reversible ones in efficiency.

Lifting techniques create beneficial probability flows in extended state spaces.

Practical implementations like shift and directed-worm algorithms are effective.

Abstract

The Markov chain Monte Carlo (MCMC) method is widely used in various fields as a powerful numerical integration technique for systems with many degrees of freedom. In MCMC methods, probabilistic state transitions can be considered as a random walk in state space, and random walks allow for sampling from complex distributions. However, paradoxically, it is necessary to carefully suppress the randomness of the random walk to improve computational efficiency. By breaking detailed balance, we can create a probability flow in the state space and perform more efficient sampling along this flow. Motivated by this idea, practical and efficient nonreversible MCMC methods have been developed over the past ten years. In particular, the lifting technique, which introduces probability flows in an extended state space, has been applied to various systems and has proven more efficient than conventional reversible updates. We review and discuss several practical approaches to implementing nonreversible MCMC methods, including the shift method in the cumulative distribution and the directed-worm algorithm.