Survival of the strictest: Stable and unstable equilibria under regularized learning with partial information

TL;DR

This paper investigates how no-regret learning algorithms, especially FTRL, converge to Nash equilibria in N-player games under various information conditions, revealing that strict equilibria are stable and attracting.

Contribution

It establishes a comprehensive equivalence between the stability of Nash equilibria and their support in the context of regularized learning algorithms under partial information.

Findings

Strict Nash equilibria are stable and attracting under no-regret learning.

The stability of an equilibrium depends on its strictness, extending evolutionary game theory results.

Provides a refinement criterion for predicting learning dynamics in games.

Abstract

In this paper, we examine the Nash equilibrium convergence properties of no-regret learning in general N-player games. For concreteness, we focus on the archetypal follow the regularized leader (FTRL) family of algorithms, and we consider the full spectrum of uncertainty that the players may encounter - from noisy, oracle-based feedback, to bandit, payoff-based information. In this general context, we establish a comprehensive equivalence between the stability of a Nash equilibrium and its support: a Nash equilibrium is stable and attracting with arbitrarily high probability if and only if it is strict (i.e., each equilibrium strategy has a unique best response). This equivalence extends existing continuous-time versions of the folk theorem of evolutionary game theory to a bona fide algorithmic learning setting, and it provides a clear refinement criterion for the prediction of the day-to-day behavior of no-regret learning in games