On the convergence, lock-in probability and sample complexity of stochastic approximation

TL;DR

This paper establishes conditions for almost sure convergence of stochastic approximation algorithms, provides a practical test for tightness, and offers improved estimates on lock-in probability and sample complexity under weaker noise assumptions.

Contribution

It introduces a simple tightness test for convergence and significantly improves sample complexity bounds with weaker noise conditions.

Findings

Convergence with probability one under tightness.

A simple test for tightness ensures convergence.

Enhanced sample complexity estimates under weaker assumptions.

Abstract

It is shown that under standard hypotheses, if stochastic approximation iterates remain tight, they converge with probability one to what their o.d.e. limit suggests. A simple test for tightness (and therefore a.s. convergence) is provided. Further, estimates on lock-in probability, i.e., the probability of convergence to a specific attractor of the o.d.e. limit given that the iterates visit its domain of attraction, and sample complexity, i.e., the number of steps needed to be within a prescribed neighborhood of the desired limit set with a prescribed probability, are also provided. The latter improve significantly upon existing results in that they require a much weaker condition on the martingale difference noise.