On Last-Iterate Convergence Beyond Zero-Sum Games

TL;DR

This paper extends last-iterate convergence results beyond zero-sum games, introducing new techniques for broader game classes and learning dynamics, with implications for equilibrium approximation and efficiency.

Contribution

It provides novel regret-based analysis and convergence guarantees for optimistic mirror descent and gradient descent in diverse game settings beyond zero-sum scenarios.

Findings

OMD has bounded second-order path lengths in certain games.

OMD achieves $O(1/ oot{2}T)$ convergence rates and $O(1)$ regret bounds.

In potential games, convergence to an $ ext{epsilon}$-equilibrium occurs in $O(1/ ext{epsilon}^2)$ iterations.

Abstract

Most existing results about \emph{last-iterate convergence} of learning dynamics are limited to two-player zero-sum games, and only apply under rigid assumptions about what dynamics the players follow. In this paper we provide new results and techniques that apply to broader families of games and learning dynamics. First, we use a regret-based analysis to show that in a class of games that includes constant-sum polymatrix and strategically zero-sum games, dynamics such as \emph{optimistic mirror descent (OMD)} have \emph{bounded second-order path lengths}, a property which holds even when players employ different algorithms and prediction mechanisms. This enables us to obtain $O(1/\sqrt{T})$ rates and optimal $O(1)$ regret bounds. Our analysis also reveals a surprising property: OMD either reaches arbitrarily close to a Nash equilibrium, or it outperforms the \emph{robust price of anarchy} in efficiency. Moreover, for potential games we establish convergence to an $\epsilon$-equilibrium after $O(1/\epsilon^2)$ iterations for mirror descent under a broad class of regularizers, as well as optimal $O(1)$ regret bounds for OMD variants. Our framework also extends to near-potential games, and unifies known analyses for distributed learning in Fisher's market model. Finally, we analyze the convergence, efficiency, and robustness of \emph{optimistic gradient descent (OGD)} in general-sum continuous games.