Stability of Gradient Learning Dynamics in Continuous Games: Scalar Action Spaces

TL;DR

This paper analyzes the stability of gradient-based learning dynamics in two-player continuous scalar games, revealing that stable equilibria often differ from Nash equilibria and are robust to learning rate changes.

Contribution

It characterizes the stability of local equilibria in gradient learning dynamics and clarifies the relationship between stability and Nash equilibria in scalar continuous games.

Findings

Stable and Nash equilibria are robust to learning rate variations.

Not all locally stable equilibria are Nash equilibria.

The spectrum of linearized dynamics predicts stability or instability.

Abstract

Learning processes in games explain how players grapple with one another in seeking an equilibrium. We study a natural model of learning based on individual gradients in two-player continuous games. In such games, the arguably natural notion of a local equilibrium is a differential Nash equilibrium. However, the set of locally exponentially stable equilibria of the learning dynamics do not necessarily coincide with the set of differential Nash equilibria of the corresponding game. To characterize this gap, we provide formal guarantees for the stability or instability of such fixed points by leveraging the spectrum of the linearized game dynamics. We provide a comprehensive understanding of scalar games and find that equilibria that are both stable and Nash are robust to variations in learning rates.