Log-density gradient covariance and automatic metric tensors for Riemann manifold Monte Carlo methods

TL;DR

This paper introduces a new metric tensor for Riemann manifold Monte Carlo methods based on log-density gradient covariance matrices, enabling automatic, flexible, and efficient sampling for complex Bayesian hierarchical models.

Contribution

It proposes a novel metric tensor derived from symmetric positive semidefinite log-density gradient covariance matrices, generalizing Fisher information for complex models and enhancing Riemann manifold Monte Carlo sampling.

Findings

The method is highly automatic and exploits model sparsity.

It performs competitively on challenging Bayesian hierarchical models.

The approach extends metric tensor construction beyond likelihoods to complex priors and latent structures.

Abstract

A metric tensor for Riemann manifold Monte Carlo particularly suited for non-linear Bayesian hierarchical models is proposed. The metric tensor is built from symmetric positive semidefinite log-density gradient covariance (LGC) matrices, which are also proposed and further explored here. The LGCs generalize the Fisher information matrix by measuring the joint information content and dependence structure of both a random variable and the parameters of said variable. Consequently, positive definite Fisher/LGC-based metric tensors may be constructed not only from the observation likelihoods as is current practice, but also from arbitrarily complicated non-linear prior/latent variable structures, provided the LGC may be derived for each conditional distribution used to construct said structures. The proposed methodology is highly automatic and allows for exploitation of any sparsity associated with the model in question. When implemented in conjunction with a Riemann manifold variant of the recently proposed numerical generalized randomized Hamiltonian Monte Carlo processes, the proposed methodology is highly competitive, in particular for the more challenging target distributions associated with Bayesian hierarchical models.