Sampling non-log-concave densities via Hessian-free high-resolution dynamics

TL;DR

This paper analyzes the convergence and acceleration of Hessian-free high-resolution (HFHR) dynamics for sampling from non-log-concave distributions, demonstrating theoretical improvements and practical efficiency over kinetic Langevin dynamics (KLD).

Contribution

It extends the theoretical understanding of HFHR dynamics to non-convex settings and shows explicit convergence rate improvements over KLD under certain conditions.

Findings

HFHR dynamics are exponentially contractive under dissipativity and smoothness assumptions.

HFHR exhibits a strictly better contraction rate than KLD with an explicit gain.

Numerical experiments confirm the efficiency and acceleration of HFHR over KLD in various Bayesian models.

Abstract

We study the problem of sampling from a target distribution $\pi(q)\propto e^{-U(q)}$ on $\mathbb{R}^d$, where $U$ can be non-convex, via the Hessian-free high-resolution (HFHR) dynamics, which is a second-order Langevin-type process that has $e^{-U(q)-\frac12|p|^2}$ as its unique invariant distribution, and it reduces to kinetic Langevin dynamics (KLD) as the resolution parameter $\alpha\to0$. The existing theory for HFHR dynamics in the literature is restricted to strongly-convex $U$, although numerical experiments are promising for non-convex settings as well. We focus on studying the convergence of HFHR dynamics when $U$ can be non-convex, which bridges a gap between theory and practice. Under a standard assumption of dissipativity and smoothness on $U$, we adopt the reflection/synchronous coupling method. This yields a Lyapunov-weighted Wasserstein distance in which the HFHR semigroup is exponentially contractive for all sufficiently small $\alpha>0$ whenever KLD is. We further show that, under an additional assumption that asymptotically $\nabla U$ has linear growth at infinity, the contraction rate for HFHR dynamics is strictly better than that of KLD, with an explicit gain. As a case study, we verify the assumptions and the resulting acceleration for three examples: a multi-well potential, Bayesian linear regression with $L^p$ regularizer and Bayesian binary classification. We conduct numerical experiments based on these examples, as well as an additional example of Bayesian logistic regression with real data processed by the neural networks, which illustrates the efficiency of the algorithms based on HFHR dynamics and verifies the acceleration and superior performance compared to KLD.