Quantifying Normality: Convergence Rate to Gaussian Limit for Stochastic Approximation and Unadjusted OU Algorithm

TL;DR

This paper provides explicit finite-time bounds on how quickly stochastic approximation algorithms converge to their Gaussian limit, improving understanding of their accuracy in practical, finite-time scenarios.

Contribution

It introduces non-asymptotic Wasserstein distance bounds for SA iterates, connecting convergence rates to the discrete Ornstein-Uhlenbeck process and adapting Stein's method for matrix-weighted sums.

Findings

Established explicit convergence rates for SA to Gaussian limit.

Derived tail bounds for SA errors at any finite time.

Connected convergence analysis to sampling literature via Ornstein-Uhlenbeck process.

Abstract

Stochastic approximation (SA) is a method for finding the root of an operator perturbed by noise. There is a rich literature establishing the asymptotic normality of rescaled SA iterates under fairly mild conditions. However, these asymptotic results do not quantify the accuracy of the Gaussian approximation in finite time. In this paper, we establish explicit non-asymptotic bounds on the Wasserstein distance between the distribution of the rescaled iterate at time k and the asymptotic Gaussian limit for various choices of step-sizes including constant and polynomially decaying. As an immediate consequence, we obtain tail bounds on the error of SA iterates at any time. We obtain the sharp rates by first studying the convergence rate of the discrete Ornstein-Uhlenbeck (O-U) process driven by general noise, whose stationary distribution is identical to the limiting Gaussian distribution of the rescaled SA iterates. We believe that this is of independent interest, given its connection to sampling literature. The analysis involves adapting Stein's method for Gaussian approximation to handle the matrix weighted sum of i.i.d. random variables. The desired finite-time bounds for SA are obtained by characterizing the error dynamics between the rescaled SA iterate and the discrete time O-U process and combining it with the convergence rate of the latter process.