Langevin Markov Chain Monte Carlo with stochastic gradients

TL;DR

This paper introduces a new discretization scheme for Langevin dynamics that uses stochastic gradients, reducing bias in Monte Carlo sampling for large datasets and providing exact results for Gaussian cases.

Contribution

It proposes a novel discretization method for underdamped Langevin dynamics with stochastic gradients, improving bias control in Monte Carlo sampling.

Findings

Bias in computed averages is second order in stepsize.

Exact results achieved for Gaussian distributions with normally distributed stochastic gradients.

Applicable to large-scale Bayesian inference and parameter estimation.

Abstract

Monte Carlo sampling techniques have broad applications in machine learning, Bayesian posterior inference, and parameter estimation. Often the target distribution takes the form of a product distribution over a dataset with a large number of entries. For sampling schemes utilizing gradient information it is cheaper for the derivative to be approximated using a random small subset of the data, introducing extra noise into the system. We present a new discretization scheme for underdamped Langevin dynamics when utilizing a stochastic (noisy) gradient. This scheme is shown to bias computed averages to second order in the stepsize while giving exact results in the special case of sampling a Gaussian distribution with a normally distributed stochastic gradient.