Last-Iterate Guarantees for Learning in Co-coercive Games

TL;DR

This paper proves finite-time last-iterate convergence guarantees for stochastic gradient descent in a broad class of co-coercive games under realistic noise conditions, extending prior results.

Contribution

It introduces the first last-iterate convergence bound for co-coercive games with non-vanishing noise, broadening understanding of learning dynamics.

Findings

Last-iterate bound of order O(log(t)/t^{1/3}) under general noise

Almost sure convergence of iterates to Nash equilibria

Time-average convergence guarantees

Abstract

We establish finite-time last-iterate guarantees for vanilla stochastic gradient descent in co-coercive games under noisy feedback. This is a broad class of games that is more general than strongly monotone games, allows for multiple Nash equilibria, and includes examples such as quadratic games with negative semidefinite interaction matrices and potential games with smooth concave potentials. Prior work in this setting has relied on relative noise models, where the noise vanishes as iterates approach equilibrium, an assumption that is often unrealistic in practice. We work instead under a substantially more general noise model in which the second moment of the noise is allowed to scale affinely with the squared norm of the iterates, an assumption natural in learning with unbounded action spaces. Under this model, we prove a last-iterate bound of order $O(\log(t)/t^{1/3})$, the first such bound for co-coercive games under non-vanishing noise. We additionally establish almost sure convergence of the iterates to the set of Nash equilibria and derive time-average convergence guarantees.