Last Iterate Convergence in Monotone Mean Field Games

TL;DR

This paper proves last-iterate convergence of a proximal-point method with KL regularization for monotone mean field games, introducing the APP algorithm that reliably finds equilibria without averaging.

Contribution

It establishes last-iterate convergence under non-strict monotonicity and proposes the APP algorithm for efficient equilibrium computation in MFGs.

Findings

Proximal-point with KL regularization converges to MFG equilibrium.

Mirror Descent achieves exponential last-iterate convergence.

APP algorithm reliably finds equilibria without time-averaging.

Abstract

In the Lasry--Lions framework, Mean-Field Games (MFGs) model interactions among an infinite number of agents. However, existing algorithms either require strict monotonicity or only guarantee the convergence of averaged iterates, as in Fictitious Play in continuous time. We address this gap with the following theoretical result. First, we prove that the last-iterated policy of a proximal-point (PP) update with KL regularization converges to an equilibrium of MFG under non-strict monotonicity. Second, we see that each PP update is equivalent to finding the equilibria of a KL-regularized MFG. We then prove that this equilibrium can be found using Mirror Descent (MD) with an exponential last-iterate convergence rate. Building on these insights, we propose the Approximate Proximal-Point ($\mathtt{APP}$) algorithm, which approximately implements the PP update via a small number of MD steps. Numerical experiments on standard benchmarks confirm that the $\mathtt{APP}$ algorithm reliably converges to the unregularized mean-field equilibrium without time-averaging.