Linear Mean-Field Games with Discounted Cost

TL;DR

This paper introduces linear mean-field games with discounted costs, analyzing their equilibrium properties and proposing an algorithm for computing equilibria with convergence guarantees in large populations.

Contribution

It formulates linear mean-field games with discounted costs, establishes their approximate Nash equilibrium properties, and develops a convergent algorithm for equilibrium computation.

Findings

Infinite population equilibrium is approximately Nash for large N.

Explicit error bounds for finite N are derived.

A convergent algorithm for computing equilibria is proposed.

Abstract

In this paper, we introduce discrete-time linear mean-field games subject to an infinite-horizon discounted-cost optimality criterion. The state space of a generic agent is a compact Borel space. At every time, each agent is randomly coupled with another agent via their dynamics and one-stage cost function, where this randomization is generated via the empirical distribution of their states (i.e., the mean-field term). Therefore, the transition probability and the one-stage cost function of each agent depend linearly on the mean-field term, which is the key distinction between classical mean-field games and linear mean-field games. Under mild assumptions, we show that the policy obtained from infinite population equilibrium is $\varepsilon(N)$-Nash when the number of agents $N$ is sufficiently large, where $\varepsilon(N)$ is an explicit function of $N$. Then, using the linear programming formulation of MDPs and the linearity of the transition probability in mean-field term, we formulate the game in the infinite population limit as a generalized Nash equilibrium problem (GNEP) and establish an algorithm for computing equilibrium with a convergence guarantee.