A Concentration Bound for Stochastic Approximation via Alekseev's Formula

TL;DR

This paper introduces a new concentration inequality for nonlinear stochastic approximation near stable equilibria, leveraging Alekseev's formula and martingale-differences, resulting in tighter bounds under weaker assumptions.

Contribution

It develops a novel approach using Alekseev's formula and a new concentration inequality to analyze nonlinear SA near stable equilibria, with improved bounds and weaker conditions.

Findings

Provides a tighter concentration bound for nonlinear SA.

Estimates hitting time and lock-in probability near LASE.

Bound holds even with non-square-summable stepsizes.

Abstract

Given an ODE and its perturbation, the Alekseev formula expresses the solutions of the latter in terms related to the former. By exploiting this formula and a new concentration inequality for martingale-differences, we develop a novel approach for analyzing nonlinear Stochastic Approximation (SA). This approach is useful for studying a SA's behaviour close to a Locally Asymptotically Stable Equilibrium (LASE) of its limiting ODE; this LASE need not be the limiting ODE's only attractor. As an application, we obtain a new concentration bound for nonlinear SA. That is, given $\epsilon >0$ and that the current iterate is in a neighbourhood of a LASE, we provide an estimate for i.) the time required to hit the $\epsilon-$ball of this LASE, and ii.) the probability that after this time the iterates are indeed within this $\epsilon-$ball and stay there thereafter. The latter estimate can also be viewed as the `lock-in' probability. Compared to related results, our concentration bound is tighter and holds under significantly weaker assumptions. In particular, our bound applies even when the stepsizes are not square-summable. Despite the weaker hypothesis, we show that the celebrated Kushner-Clark lemma continues to hold. %