Theoretical guarantees for stochastic gradient sampling methods via Gaussian convolution inequalities

TL;DR

This paper establishes first-order bounds on the bias of stochastic gradient Langevin dynamics in Wasserstein distance, using novel Gaussian convolution inequalities, improving understanding of invariant measure accuracy.

Contribution

It introduces new Gaussian convolution inequalities to derive sharp non-asymptotic bias bounds for stochastic gradient MCMC methods under minimal assumptions.

Findings

Bounds on bias are first-order in stepsize.

Provides quantitative measures of invariant measure accuracy.

Introduces Gaussian convolution inequalities of independent interest.

Abstract

We derive first-order (in the stepsize) bounds on the bias in Wasserstein distances of the invariant measure of stochastic gradient kinetic Langevin dynamics with minimal assumptions on the stochastic gradient noise. These bounds sharpen existing non-asymptotic guarantees for stochastic-gradient MCMC methods and provide a quantitative resolution of a previously open problem on invariant measure accuracy. The main technical ingredients are new Gaussian convolution inequalities controlling the Wasserstein-$p$ distance between a Gaussian convolved with a mean-zero perturbation and the Gaussian itself. We anticipate that these inequalities will be of independent interest beyond the present application.