Stabilization of stochastic approximation by step size adaptation

TL;DR

This paper introduces an adaptive step size scheme for stochastic approximation that stabilizes the iterates without projection, maintaining the same limiting behavior as traditional methods and simplifying analysis.

Contribution

It proposes a novel adaptive step size method that ensures stability in stochastic approximation without the need for projection schemes.

Findings

The scheme preserves the original limiting differential equation.

It simplifies the stability analysis by using Lyapunov functions.

The method avoids the complexities of projection-based stabilization.

Abstract

A scheme for stabilizing stochastic approximation iterates by adaptively scaling the step sizes is proposed and analyzed. This scheme leads to the same limiting differential equation as the original scheme and therefore has the same limiting behavior, while avoiding the difficulties associated with projection schemes. The proof technique requires only that the limiting o.d.e. descend a certain Lyapunov function outside an arbitrarily large bounded set.