Stochastic Optimization under Distributional Drift

TL;DR

This paper develops non-asymptotic convergence guarantees for stochastic optimization algorithms tracking evolving convex functions under distributional drift, highlighting the impact of drift, noise, and step decay schedules.

Contribution

It introduces novel bounds for stochastic algorithms under distributional drift, with insights into step decay benefits and explicit error component decoupling.

Findings

Convergence guarantees valid in expectation and high probability.

Identification of a low drift-to-noise regime improving tracking efficiency.

Numerical experiments confirming theoretical results.

Abstract

We consider the problem of minimizing a convex function that is evolving according to unknown and possibly stochastic dynamics, which may depend jointly on time and on the decision variable itself. Such problems abound in the machine learning and signal processing literature, under the names of concept drift, stochastic tracking, and performative prediction. We provide novel non-asymptotic convergence guarantees for stochastic algorithms with iterate averaging, focusing on bounds valid both in expectation and with high probability. The efficiency estimates we obtain clearly decouple the contributions of optimization error, gradient noise, and time drift. Notably, we identify a low drift-to-noise regime in which the tracking efficiency of the proximal stochastic gradient method benefits significantly from a step decay schedule. Numerical experiments illustrate our results.