Last-iterate Convergence in Regularized Graphon Mean Field Game

TL;DR

This paper establishes the first last-iterate convergence rates for mirror descent algorithms in regularized graphon mean field games, advancing theoretical understanding and empirical validation in complex multi-agent systems.

Contribution

It provides the first convergence rate analysis for mirror descent in GMFG, including finite and infinite state spaces, and demonstrates empirical performance against fictitious play.

Findings

Convergence rate of O(T^{-1/4}) in tabular GMFG with bandit feedback.

Improved convergence rate of O(T^{-1}) with full knowledge of costs and transitions.

Achieves a convergence rate of O(T^{-1/5}) in linear GMFG.

Abstract

To model complex real-world systems, such as traders in stock markets, or the dissemination of contagious diseases, graphon mean-field games (GMFG) have been proposed to model many agents. Despite the empirical success, our understanding of GMFG is limited. Popular algorithms such as mirror descent are deployed but remain unknown for their convergence properties. In this work, we give the first last-iterate convergence rate of mirror descent in regularized monotone GMFG. In tabular monotone GMFG with finite state and action spaces and under bandit feedback, we show a last-iterate convergence rate of $O(T^{-1/4})$. Moreover, when exact knowledge of costs and transitions is available, we improve this convergence rate to $O(T^{-1})$, matching the existing convergence rate observed in strongly convex games. In linear GMFG, our algorithm achieves a last-iterate convergence rate of $O(T^{-1/5})$. Finally, we verify the performance of the studied algorithms by empirically testing them against fictitious play in a variety of tasks.