$O\left(1/T\right)$ Time-Average Convergence in a Generalization of Multiagent Zero-Sum Games

TL;DR

This paper generalizes multiagent zero-sum matrix games and proves that alternating gradient descent converges to Nash equilibria at an $O(1/T)$ rate with larger fixed learning rates, outperforming optimistic gradient descent.

Contribution

It introduces a new class of multiagent games and establishes convergence guarantees for alternating gradient descent with larger fixed learning rates.

Findings

Convergence rate of $O(1/T)$ to Nash equilibria.

Larger fixed learning rates improve convergence speed.

Experimental results show strategies are closer to Nash equilibria with larger learning rates.

Abstract

We introduce a generalization of zero-sum network multiagent matrix games and prove that alternating gradient descent converges to the set of Nash equilibria at rate $O(1/T)$ for this set of games. Alternating gradient descent obtains this convergence guarantee while using fixed learning rates that are four times larger than the optimistic variant of gradient descent. Experimentally, we show with 97.5% confidence that, on average, these larger learning rates result in time-averaged strategies that are 2.585 times closer to the set of Nash equilibria than optimistic gradient descent.