Empirical Game-Theoretic Analysis for Mean Field Games

TL;DR

This paper introduces a simulation-based empirical game-theoretic approach for solving mean field games, employing iterative strategies and learning techniques to efficiently approximate Nash equilibria.

Contribution

It develops a novel EGTA framework for MFGs using query-based methods and proves convergence to Nash equilibria, improving sample efficiency and performance.

Findings

Outperforms direct FP in strategy iteration efficiency

Proves existence and convergence of NE in empirical MFGs

Enhances sample efficiency through game model learning and regularization

Abstract

We present a simulation-based approach for solution of mean field games (MFGs), using the framework of empirical game-theoretical analysis (EGTA). Our primary method employs a version of the double oracle, iteratively adding strategies based on best response to the equilibrium of the empirical MFG among strategies considered so far. We present Fictitious Play (FP) and Replicator Dynamics as two subroutines for computing the empirical game equilibrium. Each subroutine is implemented with a query-based method rather than maintaining an explicit payoff matrix as in typical EGTA methods due to a representation issue we highlight for MFGs. By introducing game model learning and regularization, we significantly improve the sample efficiency of the primary method without sacrificing the overall learning performance. Theoretically, we prove that a Nash equilibrium (NE) exists in the empirical MFG and show the convergence of iterative EGTA to NE of the full MFG with either subroutine. We test the performance of iterative EGTA in various games and show that it outperforms directly applying FP to MFGs in terms of iterations of strategy introduction.