Optimistic Gradient Descent Ascent in Zero-Sum and General-Sum Bilinear Games

TL;DR

This paper analyzes the convergence properties of Optimistic Gradient Descent Ascent (OGDA) in both zero-sum and general-sum bilinear games, providing new theoretical insights and practical algorithms for faster convergence and coordination.

Contribution

The paper extends convergence results of OGDA to zero-sum bilinear games with optimal geometric ratios and introduces OGDA for general-sum games, demonstrating improved convergence and coordination.

Findings

OGDA converges exponentially to saddle points in zero-sum games.

In some general-sum games, OGDA leads to exponential convergence to Nash or payoff divergence.

Using general-sum formulations can enhance min-max optimization algorithms.

Abstract

We study the convergence of Optimistic Gradient Descent Ascent in unconstrained bilinear games. In a first part, we consider the zero-sum case and extend previous results by Daskalakis et al. in 2018, Liang and Stokes in 2019, and others: we prove, for any payoff matrix, the exponential convergence of OGDA to a saddle point and also provide a new, optimal, geometric ratio for the convergence. We also characterize the step sizes inducing convergence, and are able to deduce the optimal step size for the speed of convergence. In a second part, we introduce OGDA for general-sum bilinear games: we show that in an interesting class of games, either OGDA converges exponentially fast to a Nash equilibrium, or the payoffs for both players converge exponentially fast to $+\infty$ (which might be interpreted as endogenous emergence of coordination, or cooperation, among players). We also give sufficient conditions for convergence of OGDA to a Nash equilibrium. These conditions are used to increase the speed of convergence of a min-max problem involving a matrix $A$, by introducing a general-sum game using the Moore-Penrose inverse matrix of $A$. This shows for the first time, at our knowledge, that general-sum games can be used to optimally improve algorithms designed for min-max problems. We finally illustrate our results on simple examples of Generative Adversarial Networks.