Functional Central Limit Theorem for Two Timescale Stochastic Approximation

TL;DR

This paper establishes a functional central limit theorem for two timescale stochastic approximation algorithms, showing their fluctuations converge to Gaussian processes, with faster iterates to a linear diffusion and slower to an ODE.

Contribution

It provides a rigorous limit theorem describing the asymptotic distribution of two timescale stochastic approximation algorithms.

Findings

Fluctuations converge to a Gaussian process.

Faster iterates follow a linear diffusion.

Slower iterates follow an ordinary differential equation.

Abstract

Two time scale stochastic approximation algorithms emulate singularly perturbed deterministic differential equations in a certain limiting sense, i.e., the interpolated iterates on each time scale approach certain differential equations in the large time limit when viewed on the `algorithmic time scale' defined by the corresponding step sizes viewed as time steps. Their fluctuations around these deterministic limits, after suitable scaling, can be shown to converge to a Gauss-Markov process in law for each time scale. This turns out to be a linear diffusion for the faster iterates and an ordinary differential equation for the slower iterates.