GIST: Gibbs self-tuning for locally adaptive Hamiltonian Monte Carlo

TL;DR

GIST introduces a flexible, Gibbs-based framework for locally adaptive Hamiltonian Monte Carlo, unifying several existing methods and providing a novel alternative for path length adaptation evaluated on complex models.

Contribution

The paper presents GIST, a new Gibbs self-tuning framework for adaptive HMC that generalizes existing algorithms and introduces a novel path length adaptation method.

Findings

GIST unifies multiple HMC variants including NUTS and randomized HMC.

The new path length adaptation method performs well on high-dimensional, ill-conditioned Gaussian models.

GIST demonstrates effective local adaptation across diverse models.

Abstract

We introduce a novel and flexible framework for constructing locally adaptive Hamiltonian Monte Carlo (HMC) samplers by Gibbs sampling the algorithm's tuning parameters conditionally based on the position and momentum at each step. For adaptively sampling path lengths, our Gibbs self-tuning (GIST) approach encompasses randomized HMC, multinomial HMC, the No-U-Turn Sampler (NUTS), and the Apogee-to-Apogee Path Sampler as special cases. We exemplify the GIST framework with a novel alternative to NUTS for locally adapting path lengths, evaluated with an exact Hamiltonian for a high-dimensional, ill-conditioned Gaussian measure and with the leapfrog integrator for a suite of diverse models.