Sampling from high-dimensional, multimodal distributions using automatically tuned, tempered Hamiltonian Monte Carlo

TL;DR

This paper introduces a tempered Hamiltonian Monte Carlo (THMC) method that automatically tunes tempering parameters, enabling efficient sampling from high-dimensional, strongly multimodal distributions, with demonstrated improvements over existing methods.

Contribution

The paper presents a novel THMC algorithm that combines tempering with HMC and features automatic tuning strategies for high-dimensional multimodal sampling.

Findings

THMC scales better with dimension than adaptive parallel tempering.

THMC effectively explores multimodal posterior distributions.

Automatic tuning reduces manual parameter adjustments.

Abstract

Hamiltonian Monte Carlo (HMC) is widely used for sampling from high dimensional target distributions with densities known up to proportionality. While HMC exhibits favorable scaling properties in high dimensions, it struggles with strongly multimodal distributions. Tempering methods are commonly used to address multimodality, but they can be difficult to tune, especially in high dimensional settings. In this study, we propose a method that combines tempering with HMC to enable efficient sampling from high dimensional, strongly multimodal distributions. Our approach simulates the dynamics of a time-varying Hamiltonian in which the temperature increases and then decreases over time. In the first phase, the simulated trajectory gradually explores low-density regions farther from the mode; the second phase guides it back toward a local mode. We develop efficient tuning strategies based on a time-scale transformation under which the Hamiltonian becomes approximately stationary. This leads to a tempered Hamiltonian Monte Carlo (THMC) algorithm with automatic tuning. We demonstrate numerically that our method scales more effectively with dimension than adaptive parallel tempering and tempered sequential Monte Carlo. Finally, we apply our THMC to sample from strongly multimodal posterior distributions arising in Bayesian inference.