Sampling with Riemannian Hamiltonian Monte Carlo in a Constrained Space

TL;DR

This paper introduces a Riemannian Hamiltonian Monte Carlo method tailored for efficient sampling of high-dimensional, constrained, and non-smooth distributions, significantly outperforming existing methods in practical applications.

Contribution

It develops a novel constrained Riemannian HMC algorithm that maintains sparsity and achieves dimension-independent mixing rates, enabling practical sampling in very high dimensions.

Findings

Achieves a 1,000-fold speed-up in sampling from a large human metabolic network.

Outperforms existing packages by orders of magnitude on benchmark datasets.

Successfully incorporates constraints into Riemannian HMC, maintaining efficiency in high-dimensional, non-smooth settings.

Abstract

We demonstrate for the first time that ill-conditioned, non-smooth, constrained distributions in very high dimension, upwards of 100,000, can be sampled efficiently $\textit{in practice}$. Our algorithm incorporates constraints into the Riemannian version of Hamiltonian Monte Carlo and maintains sparsity. This allows us to achieve a mixing rate independent of smoothness and condition numbers. On benchmark data sets in systems biology and linear programming, our algorithm outperforms existing packages by orders of magnitude. In particular, we achieve a 1,000-fold speed-up for sampling from the largest published human metabolic network (RECON3D). Our package has been incorporated into the COBRA toolbox.