Hypocoercivity properties of adaptive Langevin dynamics

TL;DR

This paper analyzes the convergence properties of Adaptive Langevin dynamics, demonstrating exponential convergence to the stationary distribution and providing a central limit theorem, with applications shown in Bayesian logistic regression on MNIST.

Contribution

The paper introduces a hypocoercivity analysis for Adaptive Langevin dynamics, establishing exponential convergence rates and a central limit theorem for the method.

Findings

Exponential convergence rate quantified in terms of key parameters.

Central limit theorem for time averages along stochastic paths.

Numerical validation on MNIST classification task.

Abstract

Adaptive Langevin dynamics is a method for sampling the Boltzmann-Gibbs distribution at prescribed temperature in cases where the potential gradient is subject to stochastic perturbation of unknown magnitude. The method replaces the friction in underdamped Langevin dynamics with a dynamical variable, updated according to a negative feedback loop control law as in the Nos\'e-Hoover thermostat. Using a hypocoercivity analysis we show that the law of Adaptive Langevin dynamics converges exponentially rapidly to the stationary distribution, with a rate that can be quantified in terms of the key parameters of the dynamics. This allows us in particular to obtain a central limit theorem with respect to the time averages computed along a stochastic path. Our theoretical findings are illustrated by numerical simulations involving classification of the MNIST data set of handwritten digits using Bayesian logistic regression.