Adaptive Riemannian Manifold Hamiltonian Monte Carlo with Hierarchical Metric

TL;DR

This paper introduces an adaptive hierarchical Riemannian manifold HMC that uses a closed-form leapfrog integrator, improving efficiency and applicability in high-dimensional Bayesian inference.

Contribution

It proposes a novel adaptive hierarchical RMHMC method with a closed-form integrator, enabling efficient sampling without requiring hierarchical structure in the target density.

Findings

Demonstrates improved empirical performance in high-dimensional Bayesian inference.

Provides an efficient implementation of hierarchical RMHMC with automatic tuning.

Enables direct use within NUTS without complex implementation challenges.

Abstract

Hamiltonian Monte Carlo (HMC) and its dynamic extensions, such as the No-U-Turn Sampler (NUTS), are powerful Markov chain Monte Carlo methods for sampling from complex, high-dimensional probability distributions. Riemannian manifold Hamiltonian Monte Carlo (RMHMC) extends HMC by allowing the mass matrix to depend on position, which can substantially improve mixing but also makes implementation considerably more challenging. In this paper, we study an adaptive hierarchical version of RMHMC that is well suited to many hierarchical sampling problems. A key feature of hierarchical RMHMC is that, unlike general RMHMC, it admits a closed-form explicit leapfrog integrator, enabling efficient implementation and direct use within dynamic HMC methods such as NUTS. We introduce an adaptive scheme that automatically tunes the parameters of the hierarchical mass matrix during simulation. Importantly, the target density need not exhibit any hierarchical or block structure; the hierarchy is instead imposed on the mass matrix as a modeling device to capture the local geometry of the target distribution. Numerical experiments demonstrate appealing empirical performance in high-dimensional Bayesian inference problems.