(Non-) asymptotic properties of Stochastic Gradient Langevin Dynamics

TL;DR

This paper analyzes the behavior of stochastic gradient Langevin dynamics (SGLD) with fixed step size, characterizing its bias and variance, and proposes modifications to improve its accuracy in large data set applications.

Contribution

It provides explicit bias characterization for fixed step size SGLD and introduces a modified version that reduces asymptotic bias due to stochastic gradient variance.

Findings

Explicit bias dependence on step size and gradient variance

Bounds on bias, variance, and mean squared error for finite iterations

Demonstration of SGLD's advantages over Euler method in large data regimes

Abstract

Applying standard Markov chain Monte Carlo (MCMC) algorithms to large data sets is computationally infeasible. The recently proposed stochastic gradient Langevin dynamics (SGLD) method circumvents this problem in three ways: it generates proposed moves using only a subset of the data, it skips the Metropolis-Hastings accept-reject step, and it uses sequences of decreasing step sizes. In \cite{TehThierryVollmerSGLD2014}, we provided the mathematical foundations for the decreasing step size SGLD, including consistency and a central limit theorem. However, in practice the SGLD is run for a relatively small number of iterations, and its step size is not decreased to zero. The present article investigates the behaviour of the SGLD with fixed step size. In particular we characterise the asymptotic bias explicitly, along with its dependence on the step size and the variance of the stochastic gradient. On that basis a modified SGLD which removes the asymptotic bias due to the variance of the stochastic gradients up to first order in the step size is derived. Moreover, we are able to obtain bounds on the finite-time bias, variance and mean squared error (MSE). The theory is illustrated with a Gaussian toy model for which the bias and the MSE for the estimation of moments can be obtained explicitly. For this toy model we study the gain of the SGLD over the standard Euler method in the limit of large data sets.