Wormhole Hamiltonian Monte Carlo

TL;DR

This paper introduces Wormhole Hamiltonian Monte Carlo, a novel Bayesian inference method that efficiently samples from high-dimensional, multimodal distributions by creating geometric connections called wormholes between modes.

Contribution

It proposes a new MCMC algorithm that uses Riemannian geometry and residual energy-based mode searching to improve sampling across isolated modes in high-dimensional spaces.

Findings

Effectively samples from multimodal distributions in high dimensions

Creates geometric 'wormholes' to connect modes and facilitate transitions

Adapts by discovering new modes without disrupting the stationary distribution

Abstract

In machine learning and statistics, probabilistic inference involving multimodal distributions is quite difficult. This is especially true in high dimensional problems, where most existing algorithms cannot easily move from one mode to another. To address this issue, we propose a novel Bayesian inference approach based on Markov Chain Monte Carlo. Our method can effectively sample from multimodal distributions, especially when the dimension is high and the modes are isolated. To this end, it exploits and modifies the Riemannian geometric properties of the target distribution to create \emph{wormholes} connecting modes in order to facilitate moving between them. Further, our proposed method uses the regeneration technique in order to adapt the algorithm by identifying new modes and updating the network of wormholes without affecting the stationary distribution. To find new modes, as opposed to rediscovering those previously identified, we employ a novel mode searching algorithm that explores a \emph{residual energy} function obtained by subtracting an approximate Gaussian mixture density (based on previously discovered modes) from the target density function.