ApproxED: Approximate exploitability descent via learned best responses

TL;DR

This paper introduces two novel methods for approximating Nash equilibria in continuous action space games by minimizing exploitability through learned best responses and ensembles, outperforming previous approaches.

Contribution

The paper presents two new algorithms that effectively minimize exploitability in continuous games using learned responses and ensembles, advancing equilibrium computation methods.

Findings

Both methods outperform prior techniques in continuous games.

The learned best-response approach effectively reduces exploitability.

Ensemble-based method improves convergence in GAN training.

Abstract

There has been substantial progress on finding game-theoretic equilibria. Most of that work has focused on games with finite, discrete action spaces. However, many games involving space, time, money, and other fine-grained quantities have continuous action spaces (or are best modeled as having such). We study the problem of finding an approximate Nash equilibrium of games with continuous action sets. The standard measure of closeness to Nash equilibrium is exploitability, which measures how much players can benefit from unilaterally changing their strategy. We propose two new methods that minimize an approximation of exploitability with respect to the strategy profile. The first method uses a learned best-response function, which takes the current strategy profile as input and outputs candidate best responses for each player. The strategy profile and best-response functions are trained simultaneously, with the former trying to minimize exploitability while the latter tries to maximize it. The second method maintains an ensemble of candidate best responses for each player. In each iteration, the best-performing elements of each ensemble are used to update the current strategy profile. The strategy profile and ensembles are simultaneously trained to minimize and maximize the approximate exploitability, respectively. We evaluate our methods on various continuous games and GAN training, showing that they outperform prior methods.