Dynamically rescaled Hamiltonian Monte Carlo for Bayesian Hierarchical Models

TL;DR

DRHMC is a fast, easy-to-implement Bayesian sampling method for hierarchical models that uses a modified parameterisation to improve sampling efficiency and exploits sparsity in computations.

Contribution

Introduces dynamically rescaled Hamiltonian Monte Carlo with constant information parameterisations for efficient Bayesian inference in hierarchical models.

Findings

Effective in models with multiple levels of latent variables

Exploits sparsity for computational efficiency

Applicable to a wide range of hierarchical models

Abstract

Dynamically rescaled Hamiltonian Monte Carlo (DRHMC) is introduced as a computationally fast and easily implemented method for performing full Bayesian analysis in hierarchical statistical models. The method relies on introducing a modified parameterisation so that the re-parameterised target distribution has close to constant scaling properties, and thus is easily sampled using standard (Euclidian metric) Hamiltonian Monte Carlo. Provided that the parameterisations of the conditional distributions specifying the hierarchical model are "constant information parameterisations" (CIP), the relation between the modified- and original parameterisation is bijective, explicitly computed and admit exploitation of sparsity in the numerical linear algebra involved. CIPs for a large catalogue of statistical models are presented, and from the catalogue, it is clear that many CIPs are currently routinely used in statistical computing. A relation between the proposed methodology and a class of explicitly integrated Riemann manifold Hamiltonian Monte Carlo methods is discussed. The methodology is illustrated on several example models, including a model for inflation rates with multiple levels of non-linearly dependent latent variables.