Stability of Gradient Learning Dynamics in Continuous Games: Vector Action Spaces

TL;DR

This paper analyzes the stability of gradient-based learning dynamics in two-player continuous games, introducing the quadratic numerical range to characterize spectrum and stability, and providing conditions for stability or instability across different game types.

Contribution

It introduces the quadratic numerical range for spectral analysis and decomposes the game Jacobian to assess stability, offering new insights into equilibrium robustness in continuous games.

Findings

All Nash equilibria are stable and robust in zero-sum games.

All stable points in potential games are Nash equilibria.

A sufficient condition for instability in general-sum games.

Abstract

Towards characterizing the optimization landscape of games, this paper analyzes the stability of gradient-based dynamics near fixed points of two-player continuous games. We introduce the quadratic numerical range as a method to characterize the spectrum of game dynamics and prove the robustness of equilibria to variations in learning rates. By decomposing the game Jacobian into symmetric and skew-symmetric components, we assess the contribution of a vector field's potential and rotational components to the stability of differential Nash equilibria. Our results show that in zero-sum games, all Nash are stable and robust; in potential games, all stable points are Nash. For general-sum games, we provide a sufficient condition for instability. We conclude with a numerical example in which learning with timescale separation results in faster convergence.