Intrepid MCMC: Metropolis-Hastings with Exploration

TL;DR

Intrepid MCMC introduces a coordinate transformation to enhance the mode exploration and convergence of Metropolis-Hastings algorithms, especially for complex, multi-modal distributions, demonstrated through various examples and a Bayesian inference application.

Contribution

The paper proposes a novel MH scheme called Intrepid MCMC that improves mode-finding and convergence by using a simple coordinate transformation, maintaining simplicity of the MH framework.

Findings

Significant improvement over vanilla MH in sampling complex distributions

Enhanced mixing behavior and convergence rates demonstrated

Effective in high-dimensional Bayesian inference problems

Abstract

In engineering examples, one often encounters the need to sample from unnormalized distributions with complex shapes that may also be implicitly defined through a physical or numerical simulation model, making it computationally expensive to evaluate the associated density function. For such cases, MCMC has proven to be an invaluable tool. Random-walk Metropolis Methods (also known as Metropolis-Hastings (MH)), in particular, are highly popular for their simplicity, flexibility, and ease of implementation. However, most MH algorithms suffer from significant limitations when attempting to sample from distributions with multiple modes (particularly disconnected ones). In this paper, we present Intrepid MCMC - a novel MH scheme that utilizes a simple coordinate transformation to significantly improve the mode-finding ability and convergence rate to the target distribution of random-walk Markov chains while retaining most of the simplicity of the vanilla MH paradigm. Through multiple examples, we showcase the improvement in the performance of Intrepid MCMC over vanilla MH for a wide variety of target distribution shapes. We also provide an analysis of the mixing behavior of the Intrepid Markov chain, as well as the efficiency of our algorithm for increasing dimensions. A thorough discussion is presented on the practical implementation of the Intrepid MCMC algorithm. Finally, its utility is highlighted through a Bayesian parameter inference problem for a two-degree-of-freedom oscillator under free vibration.