SGD with Dependent Data: Optimal Estimation, Regret, and Inference

TL;DR

This paper analyzes the performance of stochastic gradient descent (SGD) with dependent data, establishing optimal estimation, regret bounds, and asymptotic normality, even under complex dependence structures and unbounded covariates.

Contribution

It introduces a comprehensive analysis of SGD under dependent data, extending classical results to non-stationary, non-mixing, and decision-dependent scenarios, with new algorithms for sparse regression.

Findings

SGD achieves optimal estimation error and regret under dependent data.

Tail bounds remain sharp for infinite horizon settings.

Asymptotic distribution of SGD iterates is Gaussian with an $O(1/\sqrt{t})$ remainder.

Abstract

This work investigates the performance of the final iterate produced by stochastic gradient descent (SGD) under temporally dependent data. We consider two complementary sources of dependence: $(i)$ martingale-type dependence in both the covariate and noise processes, which accommodates non-stationary and non-mixing time series data, and $(ii)$ dependence induced by sequential decision making. Our formulation runs in parallel with classical notions of (local) stationarity and strong mixing, while neither framework fully subsumes the other. Remarkably, SGD is shown to automatically accommodate both independent and dependent information under a broad class of stepsize schedules and exploration rate schemes. Non-asymptotically, we show that SGD simultaneously achieves statistically optimal estimation error and regret, extending and improving existing results. In particular, our tail bounds remain sharp even for potentially infinite horizon $T=+\infty$. Asymptotically, the SGD iterates converge to a Gaussian distribution with only an $O_{\PP}(1/\sqrt{t})$ remainder, demonstrating that the supposed estimation-regret trade-off claimed in prior work can in fact be avoided. We further propose a new ``conic'' approximation of the decision region that allows the covariates to have unbounded support. For online sparse regression, we develop a new SGD-based algorithm that uses only $d$ units of storage and requires $O(d)$ flops per iteration, achieving the long term statistical optimality. Intuitively, each incoming observation contributes to estimation accuracy, while aggregated summary statistics guide support recovery.