Markov-Nash Equilibria in Mean-Field Games with Discounted Cost

TL;DR

This paper studies mean-field games with finite agents and discounted costs, establishing the existence of equilibria and their approximation properties as the number of agents grows large.

Contribution

It introduces a new Markov-Nash equilibrium concept for mean-field games and proves the existence and approximation results for large populations.

Findings

Existence of mean-field equilibrium under mild assumptions

Approximate Markov-Nash equilibrium for large N

Framework applicable to general Polish state spaces

Abstract

In this paper, we consider discrete-time dynamic games of the mean-field type with a finite number $N$ of agents subject to an infinite-horizon discounted-cost optimality criterion. The state space of each agent is a locally compact Polish space. At each time, the agents are coupled through the empirical distribution of their states, which affects both the agents' individual costs and their state transition probabilities. We introduce a new solution concept of the Markov-Nash equilibrium, under which a policy is player-by-player optimal in the class of all Markov policies. Under mild assumptions, we demonstrate the existence of a mean-field equilibrium in the infinite-population limit $N \to \infty$, and then show that the policy obtained from the mean-field equilibrium is approximately Markov-Nash when the number of agents $N$ is sufficiently large.