Continuous-Time Convergence Rates in Potential and Monotone Games

TL;DR

This paper establishes exponential convergence rates for continuous-time game dynamics like mirror descent and actor-critic in monotone and potential games, extending known convergence conditions.

Contribution

It introduces a novel relative characterization of monotone games and proves exponential convergence rates for MD and AC under these conditions.

Findings

MD converges with rate O(e^{-eta t}) in relatively strongly monotone games.

AC converges with rate O(e^{-eta t}) in games with a relatively strongly concave potential.

Simulations support the theoretical convergence rates.

Abstract

In this paper, we provide exponential rates of convergence to the interior Nash equilibrium for continuous-time dual-space game dynamics such as mirror descent (MD) and actor-critic (AC). We perform our analysis in $N$-player continuous concave games that satisfy certain monotonicity assumptions while possibly also admitting potential functions. In the first part of this paper, we provide a novel relative characterization of monotone games and show that MD and its discounted version converge with $\mathcal{O}(e^{-\beta t})$ in relatively strongly and relatively hypo-monotone games, respectively. In the second part of this paper, we specialize our results to games that admit a relatively strongly concave potential and show AC converges with $\mathcal{O}(e^{-\beta t})$. These rates extend their known convergence conditions. Simulations are performed which empirically back up our results.