A quantitative Robbins-Siegmund theorem

TL;DR

This paper develops a quantitative version of the Robbins-Siegmund theorem, providing bounds on the metastability regions in stochastic processes, which enhances understanding of convergence in stochastic optimization algorithms.

Contribution

It introduces a metastable analogue of Doob's theorem and a methodology for deriving quantitative bounds for stochastic processes based on supermartingales.

Findings

Establishes a bound on the metastability region for stochastic processes.

Provides a general methodology for quantitative convergence analysis.

Discusses practical applications of the quantitative bounds.

Abstract

The Robbins-Siegmund theorem is one of the most important results in stochastic optimization, where it is widely used to prove the convergence of stochastic algorithms. We provide a quantitative version of the theorem, establishing a bound on how far one needs to look in order to locate a region of \emph{metastability} in the sense of Tao. Our proof involves a metastable analogue of Doob's theorem for $L_1$-supermartingales along with a series of technical lemmas that make precise how quantitative information propagates through sums and products of stochastic processes. In this way, our paper establishes a general methodology for finding metastable bounds for stochastic processes that can be reduced to supermartingales, and therefore for obtaining quantitative convergence information across a broad class of stochastic algorithms whose convergence proof relies on some variation of the Robbins-Siegmund theorem. We conclude by discussing how our general quantitative result might be used in practice.