Almanac: HMC sampling with bounded velocity

TL;DR

This paper explores alternative momentum distributions in Hamiltonian Monte Carlo, such as relativistic and Student's t, which limit velocities and can enhance sampling efficiency in complex posteriors, with modest but problem-dependent improvements.

Contribution

It introduces and evaluates alternative momentum distributions for HMC that naturally bound velocities, demonstrating their potential to improve convergence and efficiency in challenging posterior geometries.

Findings

Moderately heavy-tailed momentum distributions perform best.

Velocity bounding can improve sampler robustness.

Efficiency gains are modest and problem-dependent.

Abstract

In Hamiltonian Monte Carlo sampling, the shape of the potential and the choice of the momentum distribution jointly give rise to the Hamiltonian dynamics of the sampler. An efficient sampler propagates quickly in all regions of the parameter space, so that the chain has a low autocorrelation length and the sampler has a high acceptance rate, with the goal of optimising the number of near-independent samples for given computational cost. Standard Gaussian momentum distributions allow arbitrarily large velocities, which can lead to inefficient exploration in posteriors with ridges or funnel-like geometries. We investigate alternative momentum distributions based on relativistic and Student's t kinetic energies, which naturally limit particle velocities and may improve robustness. Using Almanac, a sampler for cosmological posterior distributions of sky maps and power spectra on the sphere, we test these alternatives in both low- and high-dimensional settings. We find that the choice of parameterization and momentum distribution can improve convergence and effective sample rate, though the achievable gains are generally modest and strongly problem-dependent, reaching up to an order of magnitude in favorable cases. Among the momentum distributions that we tested, those with moderately heavy tails achieved the best balance between efficiency and stability. These results highlight the importance of sampler design and encourage future work on adaptive and self-tuning strategies for kinetic energy parameter optimization in high-dimensional settings.