Towards convergence to Nash equilibria in two-team zero-sum games

TL;DR

This paper investigates the complexity of finding Nash equilibria in two-team zero-sum games, demonstrating computational hardness, analyzing the failure of common algorithms, and proposing a new method with local convergence guarantees.

Contribution

It proves the hardness of computing Nash equilibria in these games, analyzes the limitations of existing algorithms, and introduces a novel control-theoretic method with convergence guarantees.

Findings

Computing NE in two-team zero-sum games is hard (CLS-hard).

Standard algorithms like gradient descent-ascent fail to converge in these settings.

A new control-theoretic method achieves local convergence to NE under certain conditions.

Abstract

Contemporary applications of machine learning in two-team e-sports and the superior expressivity of multi-agent generative adversarial networks raise important and overlooked theoretical questions regarding optimization in two-team games. Formally, two-team zero-sum games are defined as multi-player games where players are split into two competing sets of agents, each experiencing a utility identical to that of their teammates and opposite to that of the opposing team. We focus on the solution concept of Nash equilibria (NE). We first show that computing NE for this class of games is $\textit{hard}$ for the complexity class ${\mathrm{CLS}}$. To further examine the capabilities of online learning algorithms in games with full-information feedback, we propose a benchmark of a simple -- yet nontrivial -- family of such games. These games do not enjoy the properties used to prove convergence for relevant algorithms. In particular, we use a dynamical systems perspective to demonstrate that gradient descent-ascent, its optimistic variant, optimistic multiplicative weights update, and extra gradient fail to converge (even locally) to a Nash equilibrium. On a brighter note, we propose a first-order method that leverages control theory techniques and under some conditions enjoys last-iterate local convergence to a Nash equilibrium. We also believe our proposed method is of independent interest for general min-max optimization.