Modified Cholesky Riemann Manifold Hamiltonian Monte Carlo: Exploiting Sparsity for Fast Sampling of High-dimensional Targets

TL;DR

This paper introduces a modified Cholesky Riemann manifold Hamiltonian Monte Carlo method that leverages sparsity and log-concavity to enable efficient, high-dimensional sampling in complex Bayesian models.

Contribution

It proposes a novel symmetric positive definite scaling matrix for RMHMC derived from a modified Cholesky factorization, exploiting Hessian sparsity and log-concavity for faster sampling.

Findings

Efficient sampling of high-dimensional targets achieved.

Outperforms existing methods on challenging targets.

Successfully applied to real data in a state space model.

Abstract

Riemann manifold Hamiltonian Monte Carlo (RMHMC) has the potential to produce high-quality Markov chain Monte Carlo-output even for very challenging target distributions. To this end, a symmetric positive definite scaling matrix for RMHMC, which derives, via a modified Cholesky factorization, from the potentially indefinite negative Hessian of the target log-density is proposed. The methodology is able to exploit the sparsity of the Hessian, stemming from conditional independence modeling assumptions, and thus admit fast implementation of RMHMC even for high-dimensional target distributions. Moreover, the methodology can exploit log-concave conditional target densities, often encountered in Bayesian hierarchical models, for faster sampling and more straight forward tuning. The proposed methodology is compared to alternatives for some challenging targets, and is illustrated by applying a state space model to real data.