Several Remarks on the Numerical Integrator in Lagrangian Monte Carlo

TL;DR

This paper improves the Lagrangian Monte Carlo method by demonstrating second-order accuracy of its integrator, reducing computational costs, and showing increased robustness compared to traditional Hamiltonian methods in Bayesian inference.

Contribution

It proves the LMC integrator is second-order accurate, reduces determinant computations from four to two, and shows enhanced robustness over generalized leapfrog integrators.

Findings

LMC integrator is second-order accurate.

Reduced determinant computations from four to two.

LMC integrator is more robust to human error.

Abstract

Riemannian manifold Hamiltonian Monte Carlo (RMHMC) is a powerful method of Bayesian inference that exploits underlying geometric information of the posterior distribution in order to efficiently traverse the parameter space. However, the form of the Hamiltonian necessitates complicated numerical integrators, such as the generalized leapfrog method, that preserve the detailed balance condition. The distinguishing feature of these numerical integrators is that they involve solutions to implicitly defined equations. Lagrangian Monte Carlo (LMC) proposes to eliminate the fixed point iterations by transitioning from the Hamiltonian formalism to Lagrangian dynamics, wherein a fully explicit integrator is available. This work makes several contributions regarding the numerical integrator used in LMC. First, it has been claimed in the literature that the integrator is only first-order accurate for the Lagrangian equations of motion; to the contrary, we show that the LMC integrator enjoys second order accuracy. Second, the current conception of LMC requires four determinant computations in every step in order to maintain detailed balance; we propose a simple modification to the integration procedure in LMC in order to reduce the number of determinant computations from four to two while still retaining a fully explicit numerical integration scheme. Third, we demonstrate that the LMC integrator enjoys a certain robustness to human error that is not shared with the generalized leapfrog integrator, which can invalidate detailed balance in the latter case. We discuss these contributions within the context of several benchmark Bayesian inference tasks.