Decoupled Functional Central Limit Theorems for Two-Time-Scale Stochastic Approximation

TL;DR

This paper develops decoupled functional central limit theorems for two-time-scale stochastic approximation, providing a detailed asymptotic analysis of the joint behavior of the iterates with different update rates.

Contribution

It introduces a novel functional CLT framework for two-time-scale SA, capturing the decoupled asymptotic dynamics and inter-scale interactions.

Findings

Limiting dynamics resemble standard SA on each time scale

Coupling enters the limit through specific coefficients

Introduces an auxiliary sequence to handle interdependence

Abstract

In two-time-scale stochastic approximation (SA), two iterates are updated at different rates, governed by distinct step sizes, with each update influencing the other. Previous studies have demonstrated that the convergence rates of the error terms for these updates depend solely on their respective step sizes, a property known as decoupled convergence. However, a functional version of this decoupled convergence has not been explored. Our work fills this gap by establishing decoupled functional central limit theorems for two-time-scale SA, offering a more precise characterization of its asymptotic behavior. Our results show that, on each time scale, the limiting dynamics has the same form as in standard SA, and the coupling between the two iterates enters the limit only through the associated coefficients. To achieve these results, we leverage the martingale problem approach and establish tightness as a crucial intermediate step. Furthermore, to address the interdependence between different time scales, we introduce an innovative auxiliary sequence to eliminate the primary influence of the fast-time-scale update on the slow-time-scale update.