A Markov Jump Process for More Efficient Hamiltonian Monte Carlo

TL;DR

This paper introduces a continuous-time Markov jump process for Hamiltonian Monte Carlo, allowing for higher transition rates and improved mixing efficiency, demonstrated through spectral gap analysis and autocorrelation metrics.

Contribution

It presents a novel Hamiltonian Monte Carlo algorithm based on Markov jump processes, overcoming the traditional transition rate constraints and enhancing sampling performance.

Findings

Improved mixing demonstrated on example problems

Spectral gap analysis shows better convergence

Autocorrelation decreases faster with compute time

Abstract

In most sampling algorithms, including Hamiltonian Monte Carlo, transition rates between states correspond to the probability of making a transition in a single time step, and are constrained to be less than or equal to 1. We derive a Hamiltonian Monte Carlo algorithm using a continuous time Markov jump process, and are thus able to escape this constraint. Transition rates in a Markov jump process need only be non-negative. We demonstrate that the new algorithm leads to improved mixing for several example problems, both by evaluating the spectral gap of the Markov operator, and by computing autocorrelation as a function of compute time. We release the algorithm as an open source Python package.