Hamiltonian of polymatrix zero-sum games

TL;DR

This paper formulates the dynamics of polymatrix zero-sum games as Hamiltonian systems, revealing symmetries and conserved quantities, and introduces dissipative dynamics that ensure convergence to Nash equilibrium.

Contribution

It establishes a Hamiltonian framework for polymatrix zero-sum games, identifies their symmetries, and proposes a dissipative FTRL dynamics with convergence guarantees.

Findings

Hamiltonian formulation of polymatrix zero-sum games

Identification of symmetries and conserved quantities

Introduction of dissipative FTRL dynamics with convergence to Nash equilibrium

Abstract

The understanding of a dynamical system's properties can be significantly advanced by establishing it as a Hamiltonian system and then systematically exploring its inherent symmetries. By formulating agents' strategies and cumulative payoffs as canonically conjugate variables, we identify the Hamiltonian function that generates the dynamics of poly-matrix zero-sum games. We reveal the symmetries of our Hamiltonian and derive the associated conserved quantities, showing how the conservation of probability and the invariance of the Fenchel coupling are intrinsically encoded within the system. Furthermore, we propose the dissipation FTRL (DFTRL) dynamics by introducing a perturbation that dissipates the Fenchel coupling, proving convergence to the Nash equilibrium and linking DFTRL to last-iterate convergent algorithms. Our results highlight the potential of Hamiltonian dynamics in uncovering the structural properties of learning dynamics in games, and pave the way for broader applications of Hamiltonian dynamics in game theory and machine learning.