Hamiltonian Monte Carlo for efficient Gaussian sampling: long and random steps

TL;DR

This paper demonstrates that Hamiltonian Monte Carlo can efficiently sample high-dimensional Gaussian distributions using long, random integration times, achieving near-optimal query complexity in terms of the condition number and dimension.

Contribution

The authors introduce a novel HMC algorithm with long, random integration times that improves sampling efficiency for Gaussian distributions, surpassing fixed-time methods.

Findings

Achieves $ ilde{O}(\sqrt{\kappa} d^{1/4} \log(1/\varepsilon))$ gradient queries for Gaussian sampling.

Contrasts with lower bounds for fixed-time HMC, showing improved performance with long, random steps.

Provides theoretical analysis of HMC's efficiency in high-dimensional Gaussian sampling.

Abstract

Hamiltonian Monte Carlo (HMC) is a Markov chain algorithm for sampling from a high-dimensional distribution with density $e^{-f(x)}$, given access to the gradient of $f$. A particular case of interest is that of a $d$-dimensional Gaussian distribution with covariance matrix $\Sigma$, in which case $f(x) = x^\top \Sigma^{-1} x$. We show that HMC can sample from a distribution that is $\varepsilon$-close in total variation distance using $\widetilde{O}(\sqrt{\kappa} d^{1/4} \log(1/\varepsilon))$ gradient queries, where $\kappa$ is the condition number of $\Sigma$. Our algorithm uses long and random integration times for the Hamiltonian dynamics. This contrasts with (and was motivated by) recent results that give an $\widetilde\Omega(\kappa d^{1/2})$ query lower bound for HMC with fixed integration times, even for the Gaussian case.