Concentration bounds for two time scale stochastic approximation

TL;DR

This paper derives a high-probability concentration bound for two time scale stochastic approximation algorithms by modeling them as discretizations of singularly perturbed differential equations, extending single time scale results.

Contribution

It introduces a novel concentration bound for two time scale stochastic approximation using Alekseev's formula and martingale inequalities, expanding the theoretical understanding of these algorithms.

Findings

Provides a quantifiable high-probability bound for two time scale stochastic approximation.

Extends existing results from single to two time scale stochastic approximation.

Uses advanced mathematical tools like Alekseev's formula and martingale concentration inequalities.

Abstract

Viewing a two time scale stochastic approximation scheme as a noisy discretization of a singularly perturbed differential equation, we obtain a concentration bound for its iterates that captures its behavior with quantifiable high probability. This uses Alekseev's nonlinear variation of constants formula and a martingale concentration inequality and extends the corresponding results for single time scale stochastic approximation.