Exponential Lower Bounds for Fictitious Play in Potential Games

TL;DR

This paper demonstrates that Fictitious Play can require exponential time to reach a Nash equilibrium in potential games, even with simple two-player scenarios, highlighting limitations in its convergence rate.

Contribution

The work provides the first exponential lower bounds on the convergence time of Fictitious Play in potential games, especially for identical payoff two-player games.

Findings

Fictitious Play can take exponential time to converge in potential games.

Constructed games have a unique Nash equilibrium with all approximate equilibria close to it.

Results hold for arbitrary tie-breaking rules and simple two-agent games.

Abstract

Fictitious Play (FP) is a simple and natural dynamic for repeated play with many applications in game theory and multi-agent reinforcement learning. It was introduced by Brown (1949,1951) and its convergence properties for two-player zero-sum games was established later by Robinson (1951). Potential games Monderer and Shapley (1996b) is another class of games which exhibit the FP property (Monderer and Shapley (1996a)), i.e., FP dynamics converges to a Nash equilibrium if all agents follows it. Nevertheless, except for two-player zero-sum games and for specific instances of payoff matrices (Abernethy et al. (2021)) or for adversarial tie-breaking rules (Daskalakis and Pan (2014)), the convergence rate of FP is unknown. In this work, we focus on the rate of convergence of FP when applied to potential games and more specifically identical payoff games. We prove that FP can take exponential time (in the number of strategies) to reach a Nash equilibrium, even if the game is restricted to two agents and for arbitrary tie-breaking rules. To prove this, we recursively construct a two-player coordination game with a unique Nash equilibrium. Moreover, every approximate Nash equilibrium in the constructed game must be close to the pure Nash equilibrium in $\ell_1$-distance.