An example of multiple mean field limits in ergodic differential games

TL;DR

This paper presents a symmetric ergodic N-players differential game where the Nash equilibrium payoffs' limit set expands as N grows, despite having a unique mean field game equilibrium, contrasting finite horizon results.

Contribution

It provides a novel example demonstrating multiple mean field limits in ergodic differential games, highlighting differences from finite horizon cases.

Findings

Limit set of Nash payoffs becomes large as N increases.

Single mean field game equilibrium exists despite multiple limits.

Contrasts with finite horizon game results.

Abstract

We present an example of symmetric ergodic $N$-players differential games, played in memory strategies on the position of the players, for which the limit set, as $N\to +\infty$, of Nash equilibrium payoffs is large, although the game has a single mean field game equilibrium. This example is in sharp contrast with a result by Lacker [23] for finite horizon problems.

Equations

$$\underset{T\to +\infty }{\mathrm{limsup}}\frac{1}{T}\mathbb{E}\left[{\int }_0^T(L({\alpha }_t^i,X_t^i)+F(m_{{\mathbf{X}}_t}^{N,i}))dt\right]$$

$$m_{{\mathbf{X}}_t}^{N,i}:=\frac{1}{N-1}\sum _{j\ne i}{\delta }_{X_t^j}$$

$$d{X}_t^j={\alpha }_t^{j}dt+d{B}_t^j,$$

$$-\frac{1}{2}\Delta {\mu }_0-\mathrm{div}({\mu }_0{H}_p(D{u}_0(x),x))=0$$

$$-\frac{1}{2}\Delta u_0+H(D{u}_0,x)={\lambda }_0\mathrm{in}{\mathbb{T}}^d$$

$$H(p,x)=\underset{a\in {\mathbb{R}}^d}{\sup }-a\cdot p-L(a,x)$$

$$e_{\min }:=\underset{\mu ,\alpha }{\inf }{\int }_{{\mathbb{T}}^d}L(\alpha (x),x)\mu (dx)+F(\mu ),$$

$$-\frac{1}{2}\Delta \mu +\mathrm{div}(\mu \alpha )=0\mathrm{in}{\mathbb{R}}^d.$$

$$e\in [e_{\min },-{\lambda }_0+\max F),$$

$$e_{min}<e_{MFG}\le -{\lambda }_0+\max F$$

$$-\frac{1}{2}\sum _{j=1}^N{\Delta }_{x_j}{v}^{N,i}(\mathbf{x})+H(D_{x_i}{v}^{N,i}(\mathbf{x}),x_i)+\sum _{j\ne i}{D}_{x_j}{v}^{N,i}(\mathbf{x})\cdot H_p(D_{x_j}{v}^{N,j}(\mathbf{x}),x_j)=F(m_{\mathbf{x}}^{N,i})+{\lambda }^{N,i}$$

$${\int }_{{\mathbb{T}}^d}f(x)\pi ♯m(dx):={\int }_{{\mathbb{R}}^d}f(\pi (x))m(dx)\forall f\in C^0({\mathbb{T}}^d).$$

$$L=L(\alpha ,x)$$

$$F$$

$$F(m^{\prime })-F(m)={\int }_0^1{\int }_{{\mathbb{T}}^d}\frac{\delta F}{\delta m}((1-s)m+s{m}^{\prime },y)(m^{\prime }-m)(dy)ds\forall m,m^{\prime }\in \mathcal{P}({\mathbb{T}}^d).$$

$$F$$

$${\alpha }^i((x_j{)}_{j=1,\dots ,N}{)}_s={\alpha }^i((y_j{)}_{j=1,\dots ,N}{)}_s\text{mbox}a.e.on[0,t].$$

$$\left\{\begin{array}[]{l}\displaystyle dX^{i}_{s}=\alpha^{i}(X^{1}_{\cdot},\dots,X^{N}_{\cdot})ds+dB^{i}_{s},\qquad s\geq 0,\;i\in\{1,\dots,N\},\\ \displaystyle X^{i}_{0}=x^{i}_{0}\qquad i\in\{1,\dots,N\}.\end{array}\right.$$

$$J^i(\alpha )=\underset{T\to +\infty }{\mathrm{limsup}}\frac{1}{T}\mathbb{E}\left[{\int }_0^T(L({\alpha }_t^i,X_t^i)+F(\pi ♯m_{{\mathbf{X}}_t}^{N,i}))dt\right]$$

$$m_{{\mathbf{X}}_t}^{N,i}:=\frac{1}{N-1}\sum _{j\ne i}{\delta }_{X_t^j}.$$

$${\bar{\alpha }}^1(x^1,x^2,\dots ,x^N)={\bar{\alpha }}^1(x^1,x^{\sigma (2)},\dots ,x^{\sigma (N)})\forall x^1,\dots ,x^N\in C^0({\mathbb{R}}_{+},{\mathbb{R}}^d),$$

$${\bar{\alpha }}^i(x^1,\dots ,x^N):={\bar{\alpha }}^1(x^i,x^2,\dots ,x^{i-1},x^1,x^{i+1},\dots ,x^N),$$

$$J^i(\bar{\alpha })\le J^i({\alpha }^i,({\bar{\alpha }}^j{)}_{j\ne i})+\epsilon \forall {\alpha }^i\in \mathcal{A}$$

$$\left|J^i(\bar{\alpha })-e\right|\le \epsilon .$$

$$\left\{\begin{array}[]{l}\displaystyle-\frac{1}{2}\Delta u+H(Du,x)=F(\mu)+\lambda\qquad{\rm in}\;\mathbb{T}^{d},\\ \\ \displaystyle-\frac{1}{2}\Delta\mu-{\rm div}(\mu H_{p}(Du,x))=0\qquad{\rm in}\;\mathbb{T}^{d},\\ \displaystyle\mu\geq 0,\;\int_{\mathbb{T}^{d}}\mu=1.\end{array}\right.$$

$$-\frac{1}{2}\Delta u_0+H(D{u}_0,x)={\lambda }_0\mathrm{in}{\mathbb{T}}^d$$

$$-\frac{1}{2}\Delta {\mu }_0-\mathrm{div}({\mu }_0{H}_p(D{u}_0,x))=0\mathrm{in}{\mathbb{T}}^d.$$

$$e_{MFG}:=-{\lambda }_0+F({\mu }_0).$$

$$-{\lambda }_0={\int }_{{\mathbb{R}}^d\times {\mathbb{T}}^d}L(a,x){\nu }_0(da,dx)=\underset{\nu }{\inf }{\int }_{{\mathbb{R}}^d\times {\mathbb{T}}^d}L(a,x)\nu (da,dx)$$

$${\int }_{{\mathbb{R}}^d\times {\mathbb{T}}^d}(a\cdot D\varphi +\frac{1}{2}\Delta \varphi )\nu (da,dx)=0\forall \varphi \in {\mathcal{C}}^{\infty }({\mathbb{T}}^d).$$

Full text

Introduction

In this note we want to underline the role of information in mean field games. For this we study the limit of Nash equilibrium payoffs in ergodic $N-$player stochastic differential games as the number $N$ of players tends to infinity. Since the pioneering works by Lasry and Lions [24] (see also [20]) differential games with many agents have attracted a lot of attention under the terminology of mean field games. We also refer the reader to the monographs [3, 10]. Mean field games are nonatomic dynamic games, in which the agents interact through the population density.

Here we investigate in what extend the mean field game problem is the limit of the $N-$person differential games. This question is surprisingly difficult in general and is not completely understood so far in full generality. When, in the $N-$player game, players play in open-loop (i.e., observe only their own position but not the position of the other players), the mean field limit is a mean field game. The first result in that direction goes back to [24] in the ergodic setting (see also [1, 17] for statements in the same direction); extensions to the non Markovian setting can be found in Fischer [18] while Lacker gave a complete characterization of the limit [22] (see also [25] for an exit time problem). Note that these (often technically difficult) results are not entirely surprising since, in the N-player game as well as in the mean field game, the players do not observe the position of the other players: therefore there is no real change of nature between the $N-$player problem and the mean field game.

We are interested here in the $N-$player games in which players observe each other, the so-called closed-loop regime. In this setting, the mean field limit is much less understood and one possesses only partial results. In general, one formalizes the closed-loop Nash equilibria in the $N-$person game by a PDE (the Nash system) which describes the fact that players react in function of the current position of all the other players. The first convergence result in this setting [6] states that, in the finite horizon problem and under a suitable monotonicity assumption, the solution of the Nash system converges to a MFG equilibrium. The convergence relies on the construction of a solution to the so-called master equation, a partial differential equation stated in the space of probability measures. The result was later applied and extended to different frameworks, with similar—or closely related—techniques of proof in [2, 7, 11, 12, 13, 14].

Recently Lacker [23] introduced completely different arguments to handle the problem. He proved the convergence of the closed-loop Nash equilibria to “extended” MFG equilibria. Even more surprisingly, his result extends to generalized Markov strategies, where players can remember the past positions of the other players (memory strategies). The key remark is that, for a large number of players and “in average”, the fact that a single player deviates does not change too much the (time dependent) distribution of the players. Note that [23] holds in a set-up in which the noise of each player is non degenerate.

It is important to point out that the result of [23] cannot be extended to strategies in which players observe the controls of the other players. Indeed, the so-called Folk Theorem [4, 21] states that players can detect and “punish” a player who deviates and therefore, even when the number of players is large, the behavior of a single player completely changes the outcome of the game. A way to understand [23] is that, because of the (nondegenerate) noise, the observation of a player’s position does not give information on the fact that this player has deviated or not.

The aim of the present paper is to address a similar question for (a particular class of) ergodic differential games. Let us first recall that, in the open-loop regime, limits of Nash equilibria in the $N-$player game are MFG equilibria [1, 17, 24]. On the other hand, in the closed loop Markovian regime, the convergence problem is surprisingly open up to now, although the existence of a solution to the ergodic master equation is known [8]: Indeed in this ergodic set-up, the use of the solution to the master equation is not obvious and the technique of proof of [6] does not seem to apply. Here we concentrate on the limit of equilibria in $N-$player differential games with generalized Markov strategies. While, for the finite horizon problems, these Nash equilibria always converge to MFG equilibria [23], we show that this is no longer the case in the ergodic regime.

This means that, when the horizon becomes infinite, players can learn from the other players even if they observe their positions only. Our convergence result is reminiscent of the Folk Theorem of [4], but in a framework of an ergodic cost and in which players observe only the positions of the other players.

In order to explain more precisely our result, let us describe the framework in which we work. We consider $N-$player differential games played in strategies depending on the past positions of all the players (See Subsection 1.2 below). Player $i$ (where $i\in\{1,\dots,N\}$) minimizes an ergodic and symmetric cost of the form

[TABLE]

where, for any $j\in\{1,\dots,N\}$, $X^{j}$ is the position of player $j$ at time $t$, $\alpha^{j}$ is the control of player $j$, ${\bf X}=(X^{1},\dots,X^{N})$,

[TABLE]

is the empirical measure of all players but player $i$ and the dynamic of $X^{j}$ is just

[TABLE]

where the $B^{j}$ are independent $d-$dimensional Brownian motion. Moreover, $F:{\mathcal{P}}(\mathbb{T}^{d})\rightarrow\mathbb{R}$ is a sufficiently smooth map (where ${\mathcal{P}}(\mathbb{T}^{d})$ is the set of Borel probability measures on $\mathbb{T}^{d}$). Note that we work here with periodic data (and thus in the $d-$dimensional torus $\mathbb{T}^{d}$).

In this setting the mean field game payoff is unique and given by $e_{MFG}:=-\lambda_{0}+F(\mu_{0})$, where $\mu_{0}$ is the unique invariant measure solution to the equation

[TABLE]

and the pair $(u_{0},\lambda_{0})$ is the unique solution to the ergodic Hamilton-Jacobi equation:

[TABLE]

where

[TABLE]

(see [8]).

On the other hand, the “social cost” (i.e., the smallest cost a global planner can achieve, see [9]) is given by

[TABLE]

where the infimum is taken over all pair $(\mu,\alpha)$ where $\mu$ is the invariant measure on $\mathbb{T}^{d}$ associated with the distributed control $\alpha:\mathbb{T}^{d}\rightarrow\mathbb{R}^{d}$, i.e., satisfying the equation

[TABLE]

Our result states that, for any

[TABLE]

there exists a symmetric Nash equilibrium payoff $(e^{N},\dots,e^{N})$ in the $N-$player game such that $e^{N}$ converges to $e$. Unless $F$ is constant, the interval in (0.1) has a non empty interior:

[TABLE]

(see [9]). So the limit of the $N-$player game contains many more Nash equilibrium payoffs than the MFG one, including the social cost.

Let us underline again that our result says nothing on the convergence, as $N\rightarrow+\infty$, of the solution $(v^{N,i})$ of the $N-$player Nash system

[TABLE]

for $i=1,\dots,N$ and ${\bf x}=(x_{1},\dots,x_{N})\in(\mathbb{T}^{d})^{N}$. This convergence is, so far, an open problem.

The paper is organized as follows: in the first section, we state our assumptions and introduce the main definitions (Nash equilibria in generalized Markov strategies, mean field game equilibria, social cost). The second section is dedicated to the statement and the proof of the existence of several mean field limits.

Acknowledgement: The authors were partially supported by the ANR (Agence Nationale de la Recherche) project ANR-16-CE40-0015-01. The first author was partially supported by the Office for Naval Research Grant N00014-17-1-2095.

1 Notation, assumption and basic definitions

1.1 Notation and assumptions

Our game takes place in $\mathbb{R}^{d}$. However our data are $\mathbb{Z}^{d}-$periodic in space, which means that we mostly work in the $d-$dimensional torus $\mathbb{T}^{d}=\mathbb{R}^{d}/\mathbb{Z}^{d}$ and we denote by $\pi:\mathbb{R}^{d}\rightarrow\mathbb{T}^{d}$ the natural projection. Given a Borel probability measure $m$ on $\mathbb{R}^{d}$, we often project it into the set ${\mathcal{P}}(\mathbb{T}^{d})$ of Borel probability measures on $\mathbb{T}^{d}$ by considering $\pi\sharp m$ defined by

[TABLE]

Our problem involves the following data: The Lagrangian $L:\mathbb{R}^{d}\times\mathbb{T}^{d}\rightarrow\mathbb{R}$ satisfies, for some constant $C_{0}>0$:

[TABLE]

It will often be convenient to extend $L$ to $\mathbb{R}^{d}\times\mathbb{R}^{d}$ by setting $L(\alpha,x)=L(\alpha,\pi(x))$. The map $F:{\mathcal{P}}(\mathbb{T}^{d})\rightarrow\mathbb{R}$ satisfies

[TABLE]

Let us recall that for $F$ to be of class $C^{1}$ means that there exists a continuous map $\frac{\delta F}{\delta m}:{\mathcal{P}}(\mathbb{T}^{d})\times\mathbb{T}^{d}\rightarrow\mathbb{R}$ such that

[TABLE]

In particular, $F$ is continuous in $m$. It will be convenient to assume that

[TABLE]

Throughout the paper and for any $N\in\mathbb{N}$, the initial condition $(x^{1}_{0},\dots,x^{N}_{0})\in(\mathbb{T}^{d})^{N}$ is fixed (and actually irrelevant).

1.2 Nash equilibria in generalized Markov strategies

In the $N-$player game, players play nonanticipative strategies on the trajectory of the other players: namely, a strategy of player $i\in\{1,\dots,N\}$ is a bounded map $\alpha^{i}:\mathbb{R}_{+}\times(C^{0}(\mathbb{R}_{+},\mathbb{R}^{d}))^{N}\rightarrow\mathbb{R}^{d}$ which is Borel measurable and such that, for any $(x_{j})_{j=1,\dots,N}$ and $(y_{j})_{j=1,\dots,N}$ in $(C^{0}(\mathbb{R}_{+},\mathbb{R}^{d}))^{N}$ which coincide on $[0,t]$, we have

[TABLE]

These strategies are called “generalized Markov strategies”. We denote by ${\mathcal{A}}$ the set of generalized Markov strategies for a player (note that it does not depend on $i$ since all the players are symmetric).

For given $(\alpha^{1},\dots,\alpha^{N})\in{\mathcal{A}}^{N}$ and an initial condition ${\bf x_{0}}=(x^{1}_{0},\dots x^{N}_{0})\in(\mathbb{R}^{d})^{N}$, let us consider the SDE

[TABLE]

Using Girsanov’s theorem, we can find a filtered probability space $(\Omega,{\cal F},\mathbb{F}=({\cal F}_{t})_{t\geq 0},\mathbb{P})$ and, on this space, a couple of processes $({\bf X}_{t},B_{t})_{t\geq 0}$ with values in $\mathbb{R}^{d\times N}\times\mathbb{R}^{d\times N}$, such that ${\bf X}=(X^{1},\dots,X^{N})$ is adapted to $\mathbb{F}$, $B$ is a $\mathbb{F}$-Brownian motion and equation (1.5) is satisfied for all $i$. Moreover the couple $({\bf X},B)$ is unique in law. In what follows, as in Lacker [23], we keep in mind that for each $N$-uple of strategies, the process ${\bf X}$ is defined on a different probability space, but, for the comfort of the reader, we don’t mention this dependence in the notations.

In this game, we assume that the payoff of players takes the form of an ergodic cost of mean field type. Given a family of strategies ${\bf\alpha}=(\alpha^{1},\dots,\alpha^{N})\in{\mathcal{A}}^{N}$, the cost $J^{i}$ of player $i\in\{1,\dots,N\}$ is

[TABLE]

where ${\bf X}=(X^{1},\dots,X^{N})$ is the solution to (1.5), $m^{N,i}_{{\bf X}_{t}}$ is the empirical measure of all players but player $i$:

[TABLE]

In this setting, the definition of a symmetric Nash equilibrium payoff is the following:

Definition 1.1**.**

Fix a symmetric initial position ${\bf x_{0}}=(x_{0},\dots x_{0})\in(\mathbb{R}^{d})^{N}$. We say that $e\in\mathbb{R}$ is a symmetric Nash equilibrium payoff (in generalized Markov strategies) if, for any $\varepsilon>0$, there exists $\bar{\alpha}^{1}\in{\mathcal{A}}$ a strategy for player $1$, which is symmetric with respect to the other players:

[TABLE]

for any permutation $\sigma$ on $\{2,\dots,N\}$ and such that, if we define the strategy of player $i$ by

[TABLE]

then ${\bf\bar{\alpha}}:=(\bar{\alpha}^{1},\dots,\bar{\alpha}^{N})$ is an $\varepsilon-$Nash equilibrium with payoff close to $e$: for any $i\in\{1,\dots,N\}$,

[TABLE]

and

[TABLE]

1.3 The ergodic MFG equilibrium

As the number $N$ of players tends to infinity, one often expects that the limit of an $N-$player differential game becomes a mean field game. As our game is of ergodic type, the MFG equilibrium takes the form of the following ergodic MFG, in which the unknown are $(\lambda,u,\mu)$:

[TABLE]

It turns out that, for our problem, the unique MFG equilibrium has a very simple structure:

Proposition 1.2**.**

The unique MFG equilibrium is given by $(\lambda_{0}-F(\mu_{0}),u_{0},\mu_{0})$ where $(\lambda_{0},u_{0})$ solve the ergodic problem

[TABLE]

($u_{0}$ is unique up to constants) and $\mu_{0}$ is the unique probability measure such that

[TABLE]

In this case, the payoff of the MFG equilibrium is given by

[TABLE]

Proof.

The proof is immediate, since, $(\lambda,u,\mu):=(\lambda_{0}-F(\mu_{0}),u_{0},\mu_{0})$ satisfies the MFG system (1.7) and the solution of this system is unique because $F$, being independent of $x$, satisfies the standard Lasry-Lions monotonicity condition (see [24]).

Let us recall for later use that the measure $\nu_{0}:=(Id,-H_{p}(Du_{0}))\sharp\mu_{0}$ minimizes the energy

[TABLE]

where the infimum in the last term is computed among all the Borel probability measures on $\mathbb{R}^{d}\times\mathbb{T}^{d}$ which are closed (see [19]):

[TABLE]

1.4 The social cost

A last notion of interest in our problem is the social cost, or the cost for a global planner. It takes the form

[TABLE]

where the infimum is computed among the pairs $(m,\alpha)$, where $m\in{\mathcal{P}}(\mathbb{T}^{d})$ and $\alpha\in L^{2}_{m}(\mathbb{T}^{d},\mathbb{R}^{d})$ satisfy the constraint

[TABLE]

We will often use the fact that, if $(m,\alpha)$ is as above, then the measure $\nu:=(Id,\alpha)\sharp m$ satisfies (1.9) and therefore

[TABLE]

It is known (see [19]) that, under our continuity and growth assumption on $L$ and $F$ in (1.2) and (1.3), the problem has at least one solution $(\tilde{m},\tilde{\alpha})$ and that, under the differentiability assumption (1.3) on $F$, there exists $\tilde{u}:\mathbb{T}^{d}\rightarrow\mathbb{R}$ of class $C^{2}$ such that the pair $(\tilde{u},\tilde{m})$ satisfies the (new) MFG system

[TABLE]

with $\tilde{\alpha}(x)=-H_{p}(D\tilde{u}(x),x)$. In view of the regularity of $\tilde{u}$, $\tilde{m}$ is at least of class $C^{2}$ and is positive.

In a previous paper [9] (in a more general set-up than here, for time dependent MFGs), we have proved that there is no equality between the MFG and the social cost, unless $F$ is constant: More precisely, if $F$ satisfies (1.4), then

[TABLE]

2 The convergence result

We explain here that, for our problem, one cannot expect the convergence of all the Nash equilibria in generalized Markov strategies to the MFG equilibrium:

2.1 The main theorem

Theorem 2.1**.**

Under the assumptions (1.2), (1.3) and (1.4) on $L$ and $F$, and for any

[TABLE]

there exists a sequence of symmetric Nash equilibrium payoffs $(e^{N})$ in the $N-$player game such that

[TABLE]

Let us recall that $e_{\min}$, defined by (1.10), corresponds to the “social cost”, while $e_{\max}$, introduced in (1.12), is not smaller than the cost associated with the mean field game. In view of (1.12), Theorem 2.1 implies that the limit of symmetric Nash equilibria in generalized Markov strategies is not necessarily an MFG equilibrium. This is in sharp contrast with the finite horizon problem studied by Lacker in [23].

The construction of the symmetric Nash equilibrium payoffs $e^{N}$ is based on the “Folk Theorem” in differential games: see [4]. In general, the Folk Theorem is related to the observation of the control of the players. Surprisingly here, only the observation of the position of the other players is necessary: this is specific to the ergodic cost (and of the particular structure of our game).

2.2 Proof of the main theorem

As $F$ and $L$ are bounded below, we can assume without loss of generality that

[TABLE]

The first step consists in showing that the cost $e$ can be achieved by a suitable stationary solution $(\hat{m},\hat{\alpha})$:

Lemma 2.2**.**

There exists $(\hat{m},\hat{\alpha})$, of class $C^{1}$, with $\hat{m}$ a probability measure on $\mathbb{T}^{d}$ with a positive density, satisfying (1.11) and such that

[TABLE]

where the infimum is taken over the vector fields $\alpha\in L^{2}_{\hat{m}}(\mathbb{T}^{d},\mathbb{R}^{d})$ such that $(\hat{m},\alpha)$ satisfies (1.11).

In addition, there exists a sequence $(m^{n},\alpha^{n})$, of class $C^{1}$, with $m^{n}$ a probability measure on $\mathbb{T}^{d}$ with a positive density, satisfying (1.11) and such that

[TABLE]

Proof.

In a first step, we show that there exists $(\hat{m}^{\prime},\hat{\alpha}^{\prime})$ of class $C^{1}$, with $\hat{m}^{\prime}$ a probability measure on $\mathbb{T}^{d}$ with a positive density, satisfying (1.11) and such that

[TABLE]

where the infimum is taken over the vector fields $\alpha\in L^{2}_{\hat{m}^{\prime}}(\mathbb{T}^{d},\mathbb{R}^{d})$ such that $-\Delta\hat{m}^{\prime}+{\rm div}(\hat{m}^{\prime}\alpha)=0$ in $\mathbb{T}^{d}$. For proving this, we now build the sequence $(m^{n},\alpha^{n})$, where, for each $n$, $(m^{n},\alpha^{n})$ is a minimum of the problem:

[TABLE]

Let us recall that such a minimum exists (see Subsection 1.4). In addition, there exists $u^{n}$ such that $(u^{n},m^{n})$ solves the MFG system

[TABLE]

with $\alpha^{n}=-H_{p}(Du^{n},x)$. In particular, $(m^{n},\alpha^{n})$ is of class $C^{1}$ and $m^{n}$ has a positive density. Next we claim that

[TABLE]

Indeed, we can find another sequence $(\mu^{k})$ of smooth and positive probability densities on $\mathbb{T}^{d}$ such that $(F(\mu^{k}))$ converges to $\max F$. Let us set $\beta^{k}:=D(\log(\mu^{k}))$. Then $(\mu^{k},\beta^{k})$ satisfies the constraint (1.11) and therefore, for any $k$ and by the optimality of $(m^{n},\alpha^{n})$,

[TABLE]

Letting $k\rightarrow+\infty$ proves that $(F(m^{n}))$ converges to $\max F$. Then, recalling the characterization of $\lambda_{0}$ in (1.8), we have

[TABLE]

So, for $n$ large enough, we have

[TABLE]

Setting $(\hat{m}^{\prime},\hat{\alpha}^{\prime}):=(m^{n},\alpha^{n})$ for such a large $n$ proves the first step. We also set for later use $\hat{u}^{\prime}:=u^{n}$ and recall that $\alpha^{n}=-H_{p}(D\hat{u}^{\prime},x)$.

We now build the pair $(\hat{m},\hat{\alpha})$ required in the lemma. For $\lambda\in[0,1]$, let $m^{\lambda}$ be the unique invariant measure associated with the vector field $\alpha^{\lambda}(x):=-H_{p}(D\phi^{\lambda}(x),x)$, where $\phi^{\lambda}:=(1-\lambda)\tilde{u}+\lambda\hat{u}^{\prime}$. Note that $m^{\lambda}$ is unique, of class $C^{1}$ and has a positive density since $\alpha^{\lambda}$ is of class $C^{1}$. Moreover, $\lambda\rightarrow m^{\lambda}$ is continuous in $C^{1}$ by the same argument. Next we note that $\alpha^{\lambda}$ is a minimum of the problem:

[TABLE]

Indeed it is well-known that a vector field $\alpha$ is a minimum of this problem if and only if there exists $\phi\in H^{1}(\mathbb{T}^{d})$ such that $\alpha=-H_{p}(D\phi,x)$ and the pair $(m^{\lambda},-H_{p}(D\phi,x))$ satisfies (1.11): this is indeed the case for $\alpha^{\lambda}$ by construction.

As the map $\lambda\rightarrow\int_{\mathbb{T}^{d}}L(\alpha^{\lambda},x)m^{\lambda}+F(m^{\lambda})$ is continuous and as it is equal to $e_{\min}$ (which is not larger than $e$) for $\lambda=0$ and—by (2.16)—is not smaller than $e$ for $\lambda=1$, we can find $\lambda\in[0,1]$ such that $e=\int_{\mathbb{T}^{d}}L(\alpha^{\lambda},x)m^{\lambda}+F(m^{\lambda})$. We set $(\hat{m},\hat{\alpha}):=(m^{\lambda},\alpha^{\lambda})$ from now on and $(\hat{m},\hat{\alpha})$ satisfies the required conditions.

A second lemma shows that $\hat{\alpha}$ remains $\epsilon$-optimal in (2.15), after a sufficiently small perturbation of $\hat{m}$.

Lemma 2.3**.**

For any $\varepsilon>0$ there exists $\delta>0$ (depending on $\varepsilon$), such that, for any closed Borel probability measure $\sigma$ on $\mathbb{R}^{d}\times\mathbb{T}^{d}$ with second marginal $\mu$ and with ${\bf d}_{1}(\mu,\hat{m})\leq\delta$, one has

[TABLE]

Proof.

By the definition of $(\hat{m},\hat{\alpha})$, $\hat{\alpha}$ minimizes the quantity

[TABLE]

where the infimum is taken over the maps $\alpha\in L^{2}_{\hat{m}}(\mathbb{T}^{d})$ such that

[TABLE]

Let us now argue by contradiction and assume that there exists a sequence $(\sigma^{n})$ of closed measures, with second marginal $m^{n}$ converging to $\hat{m}$ and with

[TABLE]

In view of the coercivity of $L$ with respect to the first variable, the sequence $\sigma^{n}$ is tight and there exists a subsequence (still labelled in the same way) which converges to some measure $\sigma$ on $\mathbb{R}^{d}\times\mathbb{T}^{d}$. Note that $\sigma$ is closed (as the limit of the $(\sigma^{n})$ which are closed) and its second marginal is $\hat{m}$. Let us disintegrate $\sigma$ with respect to $\hat{m}$: $\sigma=\sigma_{x}(d\alpha)\hat{m}(dx)$ and let us set $\tilde{\alpha}(x):=\int_{\mathbb{T}^{d}}\alpha\sigma_{x}(d\alpha)$. Then $(\hat{m},\tilde{\alpha})$ satisfies (1.11) since the measure $\sigma$ is closed. In addition, by convexity of $L$ with respect to the first variable and (2.17),

[TABLE]

which contradicts the optimality of $\hat{\alpha}$.

We now build the Nash equilibrium payoff and the corresponding strategies. Let $(\hat{m},\hat{\alpha})$ and $(m^{n},\alpha^{n})$ be as in Lemma 2.2. Let us set

[TABLE]

(Recall that the empirical measure $m^{N,1}_{\bf x}$ is defined by (1.6)). By the Glivenko-Cantelli law of large numbers and (2.15),

[TABLE]

As $e<-\lambda_{0}+\max F$ and by definition of $(m^{n},\alpha^{n})$, we can choose $n$ large enough (and fixed from now on) such that, for any $N$ large enough (given again by the Glivenko-Cantelli law of large numbers),

[TABLE]

Our aim is to prove that, under the above conditions, $e^{N}$ is a Nash equilibrium payoff of the $N-$player game played in generalized Markov strategies. For this, we fix $\varepsilon>0$. Given $T,\delta>0$ to be chosen below depending on $\varepsilon$, we define the strategies $\beta^{N,T,\delta,i}$ as follows: Given $(X^{1},\dots,X^{N})\in({\mathcal{C}}^{0}(\mathbb{R}_{+},\mathbb{R}^{d}))^{N}$, we define

[TABLE]

(with the usual convention $\theta(X^{1},\dots,X^{N})=+\infty$ if the right-hand side is empty). Then we set

[TABLE]

We are going to show that, if $T$ is large enough and $\delta$ is small enough (depending on $\varepsilon$), then $(\beta^{N,T,\delta,i})$ is an $\varepsilon-$Nash equilibrium with payoff given by $e^{N}$.

Lemma 2.4**.**

The payoff of the strategies $(\beta^{N,T,\delta,i})$ is almost $e^{N}$:

[TABLE]

Proof.

Let $(X^{i}_{t})$ and $(\hat{X}^{i}_{t})$ be respectively the solutions to the systems

[TABLE]

and

[TABLE]

We can find a filtered probability space endowed with an $\mathbb{R}^{N\times d}$-valued Brownian motion on which, for all $i$, both (2.21) and (2.22) admit strong solutions $X^{i}$ and $\hat{X}^{i}$. In particular, setting $\theta=\theta(X^{1},\ldots,X^{N})$, they satisfy $X^{i}_{s}=\hat{X}^{i}_{s}$ for all $s\geq 0$ and $i$, a.s. on the event $\{s\leq\theta\}$.

Define the random time

[TABLE]

with $\sup\emptyset=0$. Since, by the ergodic Theorem, for $\mathbb{P}-$a.e. $\omega\in\Omega$,

[TABLE]

the time $\tau$ is finite a.s. . It follows that

[TABLE]

So, given a fixed $K>0$ to be chosen below, we can choose $T$ large enough, depending on $N$, $\delta$, $\varepsilon$ and $K$, such that

[TABLE]

Recalling (2.14), we have

[TABLE]

where $C=\max_{x\in\mathbb{T}^{d}}|L(\hat{\alpha}(x),x)|+|L(\alpha^{n}(x),x)|+\|F\|_{\infty}$. By (2.23), we have

[TABLE]

So

[TABLE]

if we choose $K=C$. One can show in a similar way that $J^{i}((\beta^{N,T,\delta,j}))\geq e^{N}-2\varepsilon$, which proves that (2.20) holds.

Next we estimate the cost of player $1$ (to fix the ideas) if she deviates and plays some strategy $\beta$ instead of $\beta^{N,T,\delta,1}$.

Lemma 2.5**.**

Let $\beta$ be a generalized Markov strategy for player 1. Then

[TABLE]

Proof.

Let $(X^{j})$ be the solution of the system

[TABLE]

We set $\theta:=\theta({\bf X})$ where ${\bf X}=(X^{1},\dots,X^{N})$ and chose again a probability space on which $X^{1},\ldots,X^{N}$ and $\hat{X}^{1},\ldots,\hat{X}^{N}$ are strong solutions of (2.24) and (2.22) respectively, and therefore $X^{j}_{s\wedge\theta}=\hat{X}^{j}_{s\wedge\theta}$ a.s., for all $s\geq 0$ and $j\geq 2$. So

[TABLE]

We evaluate successively all the terms in the right-hand side. For the first term, we claim that, a.s. in $\{\theta=+\infty\}$ and for $t$ large enough, we have

[TABLE]

For the second term, we are going to prove that a.s. in $\{\theta<+\infty\}$ and for $t$ large enough, we have

[TABLE]

and

[TABLE]

Proof of (2.26). By Lemma 2.3, we can choose $\delta>0$ small enough (depending on $\varepsilon$ but not on $T$) such that, for any closed Borel probability measure $\sigma$ on $\mathbb{R}^{d}\times\mathbb{T}^{d}$ with second marginal $\mu$ such that ${\bf d}_{1}(\mu,\hat{m})\leq\delta$, we have

[TABLE]

Let us recall that $\sigma$ being closed means that $\sigma$ satisfies (1.9).

We claim that, a.s. on the event $\{\theta=+\infty\}$ and for any $t$ large enough, we have

[TABLE]

To prove this, we fix $(\phi_{k})$ an enumerable and dense family of $C^{2}(\mathbb{T}^{d})$. Let $\Omega_{0}$ be the set of $\omega\in\{\theta=+\infty\}$ such that, for any $k\in\mathbb{N}$,

[TABLE]

By Doob’s inequality, $\Omega_{0}$ has a full probability in $\{\theta=+\infty\}$. Let us now argue by contradiction and assume that, for some $\omega\in\Omega_{0}$, there exists a sequence $t_{n}\rightarrow+\infty$ such that

[TABLE]

Let $\sigma_{n}=\sigma_{n}(\omega)$ be the Borel probability measure defined on $\mathbb{R}^{d}\times\mathbb{T}^{d}$ by

[TABLE]

Let us denote by $\mu_{n}$ the second marginal of $\sigma_{n}$. Then, by the definition of $\theta$ and as $\theta=+\infty$, we have ${\bf d}_{1}(\mu_{n},\hat{\mu})\leq\delta$ as soon as $t_{n}\geq T$. Note that, by coercivity of $L$ and (2.32), the sequence $\sigma_{n}$ is tight. Hence there exists a Borel probability measure $\sigma$ on $\mathbb{R}^{d}\times\mathbb{T}^{d}$ and a subsequence, denoted in the same way, such $\sigma_{n}$ converges to $\sigma$. Let $\mu$ be the second marginal of $\sigma$. Then ${\bf d}_{1}(\mu,\hat{m})\leq\delta$. Let us check that $\sigma$ is closed. Indeed, we have, for any $k\in\mathbb{N}$,

[TABLE]

which tends to [math] a.s. as $n\rightarrow+\infty$ thanks to (2.31). So, for any $k\in\mathbb{N}$,

[TABLE]

By the density of the ($\phi_{k}$), this proves that the measure $\sigma$ is closed. Letting $n\rightarrow+\infty$ in (2.32), we also have, by our convexity assumption on $L$ in (1.2),

[TABLE]

(See, e.g., Corollary 3.2.3. in [16]). This contradicts (2.29). So, for any $\omega\in\Omega_{0}$ and for $t$ large enough, (2.30) holds.

As the measure on $(\mathbb{T}^{d})^{N-1}$ defined by

[TABLE]

converges a.s. to the unique invariant measure on $(\mathbb{T}^{d})^{N-1}$ associated with the drift $(x_{2},\dots,x_{N})\rightarrow(\hat{\alpha}(x_{2}),\dots,\hat{\alpha}(x_{N}))$, which is $\hat{\mu}(dx_{2})\otimes\dots\otimes\hat{\mu}(dx_{N})$, we have, a.s. and for $t$ large enough,

[TABLE]

This is (2.26).

Proof of (2.27). We now turn to the estimate of the term $\{\theta<+\infty\}$ in the right-hand side of (2.25) and first show that (2.27) holds. For this, we argue as for the proof of (2.30). We fix $(\phi_{k})$ an enumerable and dense family of $C^{2}(\mathbb{T}^{d})$. Let $\Omega_{0}$ be the set of $\omega\in\{\theta<+\infty\}$ such that, for any $k\in\mathbb{N}$, (2.31) holds. We argue by contradiction, assuming that there exists $\omega\in\Omega_{0}$ and $t_{n}\rightarrow+\infty$ such that

[TABLE]

Exactly as above, let us define the measure $\sigma_{n}=\sigma_{n}(\omega)$ as the Borel probability measure on $\mathbb{R}^{d}\times\mathbb{T}^{d}$ such that

[TABLE]

By the coercivity of $L$ and assumption (2.33), the sequence $\sigma_{n}$ is tight and we can find $\sigma$ and a subsequence, denoted in the same way, such $\sigma_{n}$ converges to $\sigma$. We can check as above that $\sigma$ is closed. Letting $n\rightarrow+\infty$ in (2.33), we also have, by convexity of $L$ with respect to the first variable,

[TABLE]

This contradicts the characterization of $\lambda_{0}$ in (1.8) and (2.27) holds in $\Omega_{0}$.

Proof of (2.28). Next we note that, on $\{\theta<+\infty\}$, we have, for $j\neq i$,

[TABLE]

So, the measure on $(\mathbb{T}^{d})^{N-1}$ defined by

[TABLE]

converges a.s. to the unique invariant measure associated with the drift

[TABLE]

which is $m^{n}(dx_{2})\otimes\dots\otimes m^{n}(dx_{N})$. So, for $t$ large enough, (2.28) holds.

Conclusion. We now collect our estimates to evaluate the RHS of (2.25). As all the terms in the RHS of (2.25) are bounded below, we have, by Fatou and by (2.26), (2.27) and (2.28),

[TABLE]

By (2.19), this proves that $\displaystyle J^{1}(\beta,(\beta^{N,T,\delta,j})_{j\neq 1})\geq e^{N}-\varepsilon.$

Proof of Theorem 2.1.

In view of Lemma 2.4 and Lemma 2.5, the strategies $(\beta^{N,T,\delta,j})$ satisfy the conditions in Definition 1.1 with symmetric payoff $(e^{N},\cdots,e^{N})$. As, by (2.18), $e^{N}$ converges to $e$, this proves the theorem.