Adaptive step size selection for Hessian-based manifold Langevin samplers

TL;DR

This paper introduces an adaptive step size method for Hessian-based manifold Langevin samplers, improving their robustness and automatic tuning in complex models with intractable Fisher information.

Contribution

It proposes a novel adaptive step length procedure that addresses limitations of existing manifold MCMC methods in regions of near-zero curvature.

Findings

Performs well in low to moderate dimensions

Reduces need for manual tuning parameters

Enhances reliability of manifold MCMC methods

Abstract

The usage of positive definite metric tensors derived from second derivative information in the context of the simplified manifold Metropolis adjusted Langevin algorithm (MALA) is explored. A new adaptive step length procedure that resolves the shortcomings of such metric tensors in regions where the log-target has near zero curvature in some direction is proposed. The adaptive step length selection also appears to alleviate the need for different tuning parameters in transient and stationary regimes that is typical of MALA. The combination of metric tensors derived from second derivative information and adaptive step length selection constitute a large step towards developing reliable manifold MCMC methods that can be implemented automatically for models with unknown or intractable Fisher information, and even for target distributions that do not admit factorization into prior and likelihood. Through examples of low to moderate dimension, it is shown that proposed methodology performs very well relative to alternative MCMC methods.