Optimizing The Integrator Step Size for Hamiltonian Monte Carlo

TL;DR

This paper introduces a geometry-based method to optimize the step size in Hamiltonian Monte Carlo, improving its robustness and providing new diagnostics for better performance tuning.

Contribution

It develops a universal criterion for tuning the integrator step size in Hamiltonian Monte Carlo based on geometric insights, along with diagnostics for validity.

Findings

Optimal acceptance probability range expanded to 0.6-0.9

Larger acceptance rates are more robust in practice

Provides a geometry-based tuning and diagnostic framework

Abstract

Hamiltonian Monte Carlo can provide powerful inference in complex statistical problems, but ultimately its performance is sensitive to various tuning parameters. In this paper we use the underlying geometry of Hamiltonian Monte Carlo to construct a universal optimization criteria for tuning the step size of the symplectic integrator crucial to any implementation of the algorithm as well as diagnostics to monitor for any signs of invalidity. An immediate outcome of this result is that the suggested target average acceptance probability of 0.651 can be relaxed to $0.6 \lesssim a \lesssim 0.9$ with larger values more robust in practice.