Two-Scale Gradient Descent Ascent Dynamics Finds Mixed Nash Equilibria of Continuous Games: A Mean-Field Perspective

TL;DR

This paper introduces a two-scale mean-field gradient descent ascent method that converges exponentially to mixed Nash equilibria in continuous zero-sum games, even without convexity assumptions, and extends to unregularized objectives via simulated annealing.

Contribution

It develops a novel two-scale mean-field GDA algorithm with proven exponential convergence to MNE without convexity assumptions, and analyzes its annealing process for unregularized games.

Findings

Exponential convergence of two-scale Mean-Field GDA to MNE.

Convergence without convexity or concavity assumptions.

Effective annealing schedule for unregularized objectives.

Abstract

Finding the mixed Nash equilibria (MNE) of a two-player zero sum continuous game is an important and challenging problem in machine learning. A canonical algorithm to finding the MNE is the noisy gradient descent ascent method which in the infinite particle limit gives rise to the {\em Mean-Field Gradient Descent Ascent} (GDA) dynamics on the space of probability measures. In this paper, we first study the convergence of a two-scale Mean-Field GDA dynamics for finding the MNE of the entropy-regularized objective. More precisely we show that for each finite temperature (or regularization parameter), the two-scale Mean-Field GDA with a suitable {\em finite} scale ratio converges exponentially to the unique MNE without assuming the convexity or concavity of the interaction potential. The key ingredient of our proof lies in the construction of new Lyapunov functions that dissipate exponentially along the Mean-Field GDA. We further study the simulated annealing of the Mean-Field GDA dynamics. We show that with a temperature schedule that decays logarithmically in time the annealed Mean-Field GDA converges to the MNE of the original unregularized objective.