Asymptotic Time-Uniform Inference for Parameters in Averaged Stochastic Approximation

TL;DR

This paper develops time-uniform confidence sequences for parameters in stochastic approximation, providing valid inference across all time points with coverage guarantees, applicable to optimization and machine learning.

Contribution

It introduces novel asymptotic confidence sequences for SA parameters that are valid uniformly over time, including in nonlinear settings.

Findings

Confidence sequences with coverage guarantees validated through experiments

Asymptotic convergence rates of averaged iterates analyzed in both linear and nonlinear SA

Method remains valid with plug-in covariance estimators

Abstract

We study time-uniform statistical inference for parameters in stochastic approximation (SA), which encompasses a bunch of applications in optimization and machine learning. To that end, we analyze the almost-sure convergence rates of the averaged iterates to a scaled sum of Gaussians in both linear and nonlinear SA problems. We then construct three types of asymptotic confidence sequences that are valid uniformly across all times with coverage guarantees, in an asymptotic sense that the starting time is sufficiently large. These coverage guarantees remain valid if the unknown covariance matrix is replaced by its plug-in estimator, and we conduct experiments to validate our methodology.