MCMC for multi-modal distributions

TL;DR

This paper reviews the challenges of sampling from high-dimensional multimodal distributions and discusses various advanced MCMC algorithms like parallel tempering and mode jumping, demonstrating their effectiveness on synthetic and real-world examples.

Contribution

It provides a comprehensive overview of existing MCMC methods tailored for multimodal distributions, including recent state-of-the-art approaches.

Findings

Parallel tempering effectively explores multimodal spaces.

Mode jumping algorithms improve mixing in high dimensions.

Wang-Landau method adapts to complex distribution landscapes.

Abstract

We explain the fundamental challenges of sampling from multimodal distributions, particularly for high-dimensional problems. We present the major types of MCMC algorithms that are designed for this purpose, including parallel tempering, mode jumping and Wang-Landau, as well as several state-of-the-art approaches that have recently been proposed. We demonstrate these methods using both synthetic and real-world examples of multimodal distributions with discrete or continuous state spaces.