On non-uniqueness in mean field games

TL;DR

This paper investigates non-uniqueness in mean field games with a binary state space, showing multiple solutions can exist under certain conditions and identifying the entropy solution as the relevant one.

Contribution

It demonstrates the existence of multiple solutions in mean field games with anti-monotone costs and clarifies which solution is physically relevant, resolving a previous conjecture.

Findings

Multiple solutions exist when the jump rate parameter is below 1/2.

The entropy solution is the unique relevant solution when the jump rate is zero.

The paper resolves a conjecture about solution selection in mean field games.

Abstract

We analyze an $N+1$-player game and the corresponding mean field game with state space $\{0,1\}$. The transition rate of $j$-th player is the sum of his control $\alpha^j$ plus a minimum jumping rate $\eta$. Instead of working under monotonicity conditions, here we consider an anti-monotone running cost. We show that the mean field game equation may have multiple solutions if $\eta < \frac{1}{2}$. We also prove that that although multiple solutions exist, only the one coming from the entropy solution is charged (when $\eta=0$), and therefore resolve a conjecture of ArXiv: 1903.05788.

Equations

$$\mathbb{P}[Z_j(t+h)=1-i|Z_j(t)=i]=({\alpha }^j(t,\mathbf{Z}(t))+\eta )h+o(h).$$

$${\theta }^{N+1,j}(t)=\frac{1}{N}\sum _{k=1,k\ne j}^{N+1}{\delta }_{Z_k(t)=0}.$$

$$f(i,\theta )=|1-\theta -i|=\{\begin{array}{ll}1-\theta & i=0\\ \theta & i=1,\end{array}$$

$$J^{N+1}_{k}(\boldsymbol{\alpha}^{N+1})=\mathbb{E}\bigg{[}\int_{0}^{T}f(Z_{k}(t),\theta^{N+1,k}(t))+\frac{\alpha^{k}(t,\mathbf{Z}(t))}{2}dt\bigg{]}$$

$$[{\mathbf{\alpha }}^{N+1,-j};\beta {]}_k:=\{\begin{array}{ll}{\alpha }_k,& k\ne j\\ \beta ,& k=j.\end{array}$$

$$J_k^{N+1}({\mathbf{\alpha }}^{N+1})=\underset{\beta \in \mathcal{A}}{\inf }{J}_k^{N+1}([{\mathbf{\alpha }}^{N+1,-};\beta ]).$$

$$\begin{cases}-\frac{d}{dt}V^{N+1}(t,i,\theta)=f(i,\theta)-\frac{(\alpha^{N+1}_{*}(t,i,\theta))^{2}}{2}\\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ +\eta(V^{N+1}(t,1-i,\theta)-V^{N+1}(t,i,\theta))\\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ +N(1-\theta)\bigg{(}\alpha^{N+1}_{*}(t,1,\theta+\frac{1-i}{N})+\eta\bigg{)}(V^{N+1}(t,1,\theta+\frac{1}{N})-V^{N+1}(t,1,\theta))\\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ +N\theta\bigg{(}\alpha^{N+1}_{*}(t,0,\theta-\frac{i}{N})+\eta\bigg{)}(V^{N+1}(t,1,\theta-\frac{1}{N})-V^{N+1}(t,1,\theta)),\\ V^{N+1}(T,i,\theta)=0,\\ \end{cases}$$

$$a_{\ast }^{N+1}(t,i,\theta )=(V^{N+1}(t,i,\theta )-V^{N+1}(t,1-i,\theta ){)}_{+}.$$

$$\{\begin{array}{l}\frac{d}{dt}\theta (t)=(1-\theta (t))((u(t,1)-u(t,0){)}_{+}+\eta )-\theta (t)((u(t,0)-u(t,1){)}_{+}+\eta ),\\ -\frac{d}{dt}u(t,i)=f(i,\theta )-\eta (u(t,i)-u(t,1-i))-\frac{((u(t,i)-u(t,1-i){)}_{+}{)}^2}{2},\\ \theta (0)=\bar{\theta },\\ u(T,i)=0,\end{array}$$

$$\{\begin{array}{l}-\frac{\partial }{\partial t}U(t,i,\theta )=f(i,\theta )-\frac{[(U(t,i,\theta )-U(t,1-i,\theta {)}_{+}{]}^2}{2}+\eta (U(t,1-i,\theta )-U(t,i,\theta ))\\ +\frac{\partial }{\partial \theta }U(t,i,\theta )((U(t,1,\theta )-U(t,0,\theta {)}_{+}+\eta )(1-\theta )\\ -\frac{\partial }{\partial \theta }U(t,i,\theta )((U(t,0,\theta )-U(t,1,\theta {)}_{+}+\eta )\theta ,\\ U(T,i,\theta )=0,\end{array}$$

$$\sum _{i=0,1}(-1{)}^i(f(i,\theta )-f(i,{\theta }^{{}^{\prime }}))(\theta -{\theta }^{{}^{\prime }})\ge 0,$$

$$y(t)=u(t,1)-u(t,0),x(t)=2\theta (t)-1,$$

$$\{\begin{array}{l}\frac{d}{dt}x=y-x|y|-2\eta x\\ -\frac{d}{dt}y=x-\frac{1}{2}y|y|-2\eta y\\ y(T)=0,x(0)=2\bar{\theta }-1.\end{array}$$

$$x=\frac{1}{2}y|y|+2\eta y-\frac{d}{dt}y.$$

$$\frac{d^2}{d{t}^2}y+y-\frac{1}{2}{y}^3-3\eta |y|y-4{\eta }^{2}y=0.$$

$$\{\begin{array}{l}\frac{d^2}{d{t}^2}y+y-\frac{1}{2}{y}^3-3\eta |y|y-4{\eta }^{2}y=0\\ \frac{1}{2}y(T)|y(T)|+2\eta y(T)+\frac{d}{dt}y(T)=x(T)=2\bar{\theta }-1\\ y(0)=0.\end{array}$$

$$x_v(t):=\frac{1}{2}{y}_v(t)|y_v(t)|+2\eta y_v(T)+\frac{d}{dt}{y}_v(t)$$

$$\frac{d^{2}y}{dt^{2}}=\frac{d}{dt}\bigg{(}\frac{1}{2}(\frac{dy}{dt})^{2}\bigg{)}\frac{dt}{dy}=\frac{d}{dy}\bigg{(}\frac{1}{2}(\frac{dt}{dy})^{-2}\bigg{)}.$$

$$\frac{dt}{dy}=\pm \frac{1}{\sqrt{G(y)+v^2}},$$

$$G^{{}^{\prime }}(y)=y^3+6\eta y^2+8{\eta }^{2}y-2y=y(y+3\eta -\sqrt{{\eta }^2+2})(y+3\eta +\sqrt{{\eta }^2+2}).$$

$$T(v):={\int }_0^{y(v)}\frac{dz}{\sqrt{G(z)+v^2}},v\in (0,v_0),$$

$$t=\mathrm{sign}(v){\int }_0^y\frac{dz}{\sqrt{G(z)+v^2}}.$$

$$t={\int }_0^{y_v(t)}\frac{dz}{\sqrt{G(z)+v^2}}.$$

$$y_v(t)=\{\begin{array}{ll}{y}_v(t-4kT(v))& t\in [4kT(v),(4k+1)T(v)),\\ y_v((4k+2)T(v)-t)& t\in [(4k+1)T(v),(4k+2)T(v)),\\ -y_v(t-(4k+2)T(v))& t\in [(4k+2)T(v),(4k+3)T(v)),\\ -y_v((4k+4)T(v)-t)& t\in [(4k+3)T(v),(4k+4)T(v)).\end{array}$$

$${\int }_0^{+\infty }\frac{dz}{\sqrt{G(z)+u^2}}=T.$$

$$T={\int }_0^{y_v(T)}\frac{dz}{\sqrt{G(z)+v^2}},$$

$$y_{v_1}(T)<y_{v_2}(T),\frac{d}{dt}{y}_{v_1}(T)<\frac{d}{dt}{y}_{v_2}(T),$$

$$H(v):={\int }_v^{y(v)}\frac{dz}{\sqrt{G(z)+v^2}}.$$

$$T(v)={\text{bigintss}}_0^1\frac{dp}{\sqrt{\frac{G(y(v)p)}{y(v{)}^2}+\frac{v^2}{y(v{)}^2}}}={\text{bigintss}}_0^1\frac{dp}{\sqrt{\frac{1}{4}y(v{)}^2{p}^4+2\eta y(v)p^3+(4{\eta }^2-1)p^2+\frac{v^2}{y(v{)}^2}}}.$$

Full text

This research was supported in part by the National Science Foundation under grants DMS-1613170.

1. Introduction

The theory of mean field games (MFGs) was introduced recently (2006-2007) independently by Lasry, Lions (see [13], [14], [15]) and Caines, Huang, Malhamé (see [11], [12]). It is an analysis of limit models for symmetric weakly interacting $N+1$-player differential games (see e.g. [3], [4]). The solution of MFGs provides an approximated Nash Equilibrium. It also under some conditions follows that MFGs are limit points of $N+1$-player Nash equilibria.

The influential work [2] by Cardaliaguet, Delarue, Lasry, and Lions established the convergence of closed loop equilibria using the the so-called master equation, which is a partial differential equation with terminal conditions whose variable are time, state and measure. It is known that under the monotonicity condition, the master equation possess a unique solution, which is used to show the above convergence. A similar analysis was carried in finite state mean field games by Bayraktar and Cohen [1] and Cecchin and Pelino [5] independently obtain the above convergence result (as well as the the analysis of its fluctuations).

In this paper, we consider a case when the monotonicity assumption is not satisfied and resolve a conjecture of [10], in which a two-state mean field game with Markov feedback strategies is analyzed. In this game the transition rate of each player is the sum of his control and a background jump rate $\eta\geq 0$. Supposing an anti-monotone running cost (follow the crowd game), [10] poses a conjecture on the nature of the limits of $N+1$-player Nash equilibrium. We proceed by using similar techniques to [6], which considers an anti-monotone terminal condition. In particular, we again rely on the entropy solution of the master equation to prove the convergence and show that the limit of $N+1$-player Nash equilibrium selects the unique mean field equilibrium induced by this entropy solution. In [6], they showed that the mean field game equation has at most three equations, while in our model if $\eta<\frac{1}{2}$, the number of solutions is increasing with time horizon and can be arbitrarily large. Also, the entropy solution in our case cannot be written down explicitly, and so we need to construct using the characteristics and check that it is entropic. For numerical methods towards the convergence of $N+1$ player games to entropy solution, we refer readers to the work of Gomes et al. [8]. Let us mention the recent work by [7], where they study linear-quadratic mean field games in the diffusion setting. To re-establish the uniqueness of MFG solutions, they add a common noise and prove that the limit of MFG solutions as noise tends to zero is just the solution induced by the entropy solution of the master equation without common noise.

The paper is organized as follows. In Section 2, we introduce the $N+1$-player game we are considering, and introduce the equations characterizing the mean field equilibria. In Section 3, we show that the forward backward equation characterizing the mean field game possesses a unique solution if $\eta\geq\frac{1}{2}$, may have multiple solutions if $\eta<\frac{1}{2}$. Furthermore, we also determine the number of solutions. In Section 4, we explicitly find the entropy solution of the master equation. In Section 5, we show that if $\eta=0$ each player in the $N+1$-player game will follow the majority and briefly present that the optimal trajectories of $N+1$-player game converges to the optimal trajectory induced by the entropy solution of the master equation.

2. Two states mean field games

We consider the $N+1$-players game with state space $\Sigma=\{0,1\}$, and denote the state of players by $\mathbf{Z}(t):=(Z_{j}(t))_{j=1}^{N+1}$, which evolves as controlled Markov processes. The jump rate of $Z_{j}(t)$ is given by $\alpha^{j}(t,\mathbf{Z}(t))+\eta$, where $\alpha^{j}:[0,T]\times\Sigma^{N+1}\to[0,+\infty)$ is the control of player $j$ and $\eta\geq 0$ is the minimum jump rate, i.e.,

[TABLE]

Denote by $\mathcal{A}$ the collection of all the measurable and locally integrable functions $[0,T]\times\Sigma^{N+1}\to[0,+\infty)$, and by $\boldsymbol{\alpha}^{N+1}=(\alpha^{1},\dotso,\alpha^{N+1})\in\mathcal{A}^{N+1}$ the control of all players. It is can be easily seen that the law of Markov process is determined by the control vector $\boldsymbol{\alpha}^{N+1}$.

Let the empirical measure of player $j$ at time $t$ to be

[TABLE]

Then given the running cost function

[TABLE]

the control vector $\boldsymbol{\alpha}^{N+1}\in\mathcal{A}^{N+1}$ and it is associated Markov process $(\mathbf{Z}(t))_{0\leq t\leq T}$, the objective function of the $k$-th player is defined by

[TABLE]

For a control vector $\boldsymbol{\alpha}^{N+1}\in\mathcal{A}^{N+1}$ and $\beta\in\mathcal{A}$, define the perturbed control vector by

[TABLE]

Definition 2.1**.**

A control vector $\boldsymbol{\alpha}^{N+1}\in\mathcal{A}^{N+1}$ is a Nash Equilibrium if for any $k=1,\dotso,N+1$

[TABLE]

To find the Nash equilibrium, it is standard to solve its corresponding Hamilton-Jacobi equations for value functions $V^{N+1}(t,i,\theta),i=0,1$ (see e.g. [9]).

[TABLE]

where the optimal control is given by

[TABLE]

It is also easy to write down the corresponding mean field game equation,

[TABLE]

and see e.g. [9] and the corresponding master equation, the corresponding master equation,

[TABLE]

see Bayraktar, Cohen [1] and Cecchin, Pelino [5]. Recall from the latter two references that the uniqueness of (MFG) and (ME) is guaranteed by the so-called monotonicity condition, i.e., for every $\theta,\theta^{{}^{\prime}}\in[0,1]$,

[TABLE]

which does not hold true with our choice of running cost.

3. non-uniqueness

We show that the mean field equations (MFG) may have multiple solutions. Taking

[TABLE]

then $\eqref{MFG}$ becomes

[TABLE]

The second one of (3.1) is equivalent to

[TABLE]

Taking derivative with respect to $t$ in (3.2) and in conjunction with (3.1), we obtain

[TABLE]

For simplicity, we time reverse the system and try to solve

[TABLE]

Since (3.4) contains only the $y$ variable, it can be uniquely solved if imposing the initial conditions $y(0)=0,\frac{d}{dt}y(0)=v$, and we denote its $\mathcal{C}^{1}$ solution as $y_{v}(.)$. Therefore the number of solutions to (3.4) is just the number of initial velocity $v$ such that $2\bar{\theta}-1=x_{v}(T)$, where for any $t\geq 0$

[TABLE]

We rewrite the differential equation as a derivative with respect to $y$ instead of $t$, i.e.,

[TABLE]

We can therefore get an implicit solution

[TABLE]

where $G(y)=\frac{1}{4}y^{4}+2\eta|y|^{3}+4{\eta}^{2}y^{2}-y^{2}.$

When $y\geq 0$, the first order derivative of $G$ is

[TABLE]

It is then easy to conclude the following results

• •

If $\eta\geq\frac{1}{2}$, the function $G(y)$ is strictly increasing for $y\geq 0$;

• •

If $0\leq\eta<\frac{1}{2}$, the function $G(y)$ decreases on the interval $[0,\sqrt{{\eta}^{2}+2}-3\eta]$ and increases on the interval $[\sqrt{{\eta}^{2}+2}-3\eta,+\infty)$;

• •

If $\eta<\frac{1}{2},|v|<v_{0}$, the function $G(y)+v^{2}$ maybe negative for some $y\in\mathbb{R}$. Let us denote by $y(v)$ the smallest positive root of $G(y)+v^{2}=0$. Since the function $y\mapsto G(y)$ first decreases to $-v_{0}^{2}$ over the interval $[0,\sqrt{{\eta}^{2}+2}-3\eta]$, and then increasing to $+\infty$ over the interval $[\sqrt{{\eta}^{2}+2}-3\eta,+\infty)$, we know that the function $y\mapsto G(y)+v^{2}$ decreases over $[0,y(v))$ and crosses [math] at $y(v)$, which implies that $y(v)$ is a simple root.

Let $v_{0}:=\sqrt{-G(\sqrt{{\eta}^{2}+2}-3\eta)}$ if $\eta<\frac{1}{2}$. and

[TABLE]

whose role will be clear in the next result.

Lemma 3.1**.**

The following properties hold for solutions $y_{v}(.)$,

• •

$y_{v}(.)$* is strictly increasing if $v>0$, strictly decreasing if $v<0$, identically [math] if $v=0$;*

• •

If either $\eta\geq\frac{1}{2},v\in\mathbb{R}$ or $\eta<\frac{1}{2},|v|\geq v_{0}$, then the solution $y_{v}(t)<+\infty$ if and only if $t<\int_{0}^{+\infty}\frac{dz}{\sqrt{G(z)+v^{2}}}$. Furthermore, $y_{v}(.)$ is strictly increasing if $v>0$, strictly decreasing if $v<0$;

• •

If $\eta<\frac{1}{2},|v|\in(0,v_{0})$, the solution $y_{v}(.)$ is a periodic function.

Proof.

The first statement is clear. We prove the rest by writing down the unique $\mathcal{C}^{1}$ solution explicitly.

If either $\eta\geq\frac{1}{2},v\in\mathbb{R}$ or $\eta<\frac{1}{2},|v|\geq v_{0}$, then $G(z)+v^{2}\geq 0$ for any $z\in\mathbb{R}$ and thus we obtain from (3.6) that

[TABLE]

Since the function $y\mapsto\int_{0}^{y}\frac{dz}{\sqrt{G(z)+v^{2}}}$ is strictly increasing, for any $t<\int_{0}^{+\infty}\frac{dz}{\sqrt{G(z)+v^{2}}}$, we can find a unique $y_{v}(t)$ such that

[TABLE]

It can be seen that the function $t\mapsto y_{v}(t)$ is $\mathcal{C}^{1}$, and therefore is the unique solution to (3.4).

Since $G(y_{v}(t))+v^{2}$ is always nonnegative, the solution $y_{v}(t)$ must oscillate between $[-y(v),y(v)]$. For any $0\leq t\leq T(v)$, there exists a unique $y_{v}(t)$ such that

[TABLE]

Define a periodic function, still denoted by $y_{v}(.)$,

[TABLE]

It can be easily seen that $y_{v}(t)$ is the unique $\mathcal{C}^{1}$ solution to (3.4). ∎

Proposition 3.1**.**

If $\eta\geq\frac{1}{2}$, then $x_{v}(T)$ is strictly increasing with respect to $v$ and therefore (3.4) has unique solution.

Proof.

It can be seen that both of the equation (3.4) and the function $v\mapsto x_{v}(T)$ are odd. Therefore $y_{-v}(.)=-y_{v}(.)$, $x_{-v}(T)=-x_{v}(T)$, and we only need to prove the proposition for $v\geq 0$.

The strictly decreasing function $v\mapsto\int_{0}^{+\infty}\frac{dz}{\sqrt{G(z)+v^{2}}}$ approaches $+\infty$ as $v\to 0$, approaches [math] as $v\to+\infty$. Therefore any positive $T$ there exists a unique $u>0$ such that

[TABLE]

As a result of Lemma 3.1, the solution $y_{v}(.)$ is finite at $T$ if and only if $v<u$, and there exists a unique $y_{v}(T)>0$ such that

[TABLE]

and also $\frac{dy_{v}}{dt}|_{T}=\sqrt{G(y_{v}(T))+v^{2}}.$ Suppose $0\leq v_{1}<v_{2}<u$. Due to the fact that $G(z)+v_{1}^{2}<G(z)+v_{2}^{2},\forall z\in\mathbb{R}$, we obtain

[TABLE]

from which we can conclude $x_{v_{1}}(T)<x_{v_{2}}(T)$. As a result of $\lim\limits_{v\to u}y_{v}(T)=+\infty$, we obtain $\lim\limits_{v\to u}x_{v}(T)=+\infty$, and thus there exists a unique solution to (3.4) for any $2\bar{\theta}-1\in\mathbb{R}$.

∎

As a result of the above proposition, the mean field equation (3.1) may have multiple solutions only if $\eta<\frac{1}{2}$. To find the number of solutions, we study the period of $y_{v}(.)$ in the following lemma. Note that since $y_{-v}(t)=-y_{v}(t)$ and $y_{0}(t)=0$, it suffices for us to consider the period of $y_{v}(.)$ for $v\in(0,v_{0})$.

Lemma 3.2**.**

Suppose $0\leq\eta<\frac{1}{2}$, $v\in(0,v_{0})$, and $y(v)$ is the smallest postive root of $z\mapsto G(z)+v^{2}$. Recall (3.7) and define

[TABLE]

Take $T(v)=T(-v),H(v)=H(-v)$ if $v\in(-v_{0},0)$. Then both $T(.)$ and $H(.)$ are increasing with respect to $v$ over the interval $(0,v_{0})$, and $\lim\limits_{v\to v_{0}}T(v)=+\infty.$

Proof.

By the definition, we have $G(y)+v^{2}=(\frac{y^{2}}{2}+2\eta|y|)^{2}+v^{2}-y^{2}$, from which we can conclude that $y(v)\geq v$, and therefore $H(v)$ is positive.

By change of variable $p=\frac{z}{y(v)}$, we obtain

[TABLE]

Denote the square of the bottom of the integrand by $P(v,p)$, i.e.,

[TABLE]

To prove $T(v)$ is increasing, it suffices to show that $P(v,p)$ is decreasing with respect to $v$ for any fixed $p\in[0,1]$.

Since $y(v)$ is an increasing function of $v$, the derivative $\frac{dP}{dv}(v,p)$ is no larger than $\frac{dP}{dv}(v,1)$, which is equal to [math] according to the definition of $y(v)$,

[TABLE]

Therefore $P(v_{1},p)\geq P(v_{2},p)$ for any $p\in[0,1],0<v_{1}<v_{2}<v_{0}$.

We can also rewrite $H(v)$ as

[TABLE]

and it is enough to show that $v\mapsto\frac{v}{y(v)}$ is decreasing. Taking derivative of the following equation with respect to $v$,

[TABLE]

we get $\frac{dy(v)}{dv}=-\frac{2v}{G^{{}^{\prime}}(y(v))},$ and thus

[TABLE]

As a result of $\frac{dy(v)}{dv}\geq 0$, we obtain that $G^{{}^{\prime}}(y(v))<0$ and $\frac{d}{dv}(\frac{v}{y(v)})\leq 0$ is equivalent to $G^{{}^{\prime}}(y(v))y(v)+2v^{2}\geq 0.$ We conclude our claim by the following computation,

[TABLE]

In the end, it can be seen that the function $z\mapsto G(z)+v_{0}^{2}$ is always positive over the interval $[0,+\infty)$ and only attains [math] at $z=\sqrt{{\eta}^{2}+2}-3\eta$. Since $G(z)+v_{0}^{2}$ is a polynomial, we obtain that $y(v_{0})=\sqrt{{\eta}^{2}+2}-3\eta$, $(z-\sqrt{{\eta}^{2}+2}+3\eta)^{2}$ is a factor of $G(z)+v_{0}^{2}$, and hence

[TABLE]

∎

For each $k\in\mathbb{N}$, define $T_{k}(v):=(2k-1)T(v)+H(v)$ if $|v|\in(0,v_{0})$, and $T_{k}(v):=+\infty$ if $|v|>v_{0}$. Now we show that for $v\not=0$, $\{T_{k}(v):k\in\mathbb{N}\}$ is the set of times $T$ such that $x_{v}(T)$ attains [math] ($T_{k}(v)=+\infty$ for $|v|\geq v_{0}$ simply implies that $x_{v}(t)$ never reaches [math] for those $v$). As a result of Lemma 3.1, the function $x_{v}(T)$ can equal to [math] only if $\eta<\frac{1}{2},|v|\in(0,v_{0})$ or $v=0$. Setting $x_{v}(T)=0$, by (3.5) we get

[TABLE]

Moving the last term to the left, taking square of both sides and plugging in the formula of $G(y)$, it becomes

[TABLE]

which is equivalent to $v^{2}-(y_{v}(T))^{2}=0$. Therefore we obtain that $|y_{v}(T)|=v,\operatorname{sign}(y_{v}(T))=-\operatorname{sign}(\frac{d}{dt}y_{v}(T))$, from which we conclude that $x_{v}(T)=0$ if and only if $T=T_{k}(v)$ or $v=0$.

Therefore $T_{1}(v)$ is the first time $x_{v}(t)$ reaches [math]. Taking $T_{k}(0+):=\lim\limits_{v\downarrow 0}T_{k}(v)$, it can be seen that for $t\leq T_{1}(0+),v\not=0$, we have $x_{v}(t)\not=0$. Before computing the number of solutions, we still need one more result, which is also important for us to construct the entropy solution of the master equation in the next section.

Lemma 3.3**.**

Suppose $\eta<\frac{1}{2}$. Then for any $(x,t)\in\mathbb{R}\times\mathbb{R}_{+}\setminus\{0\}\times\mathbb{R}_{+}$, there exists a unique $v(x,t)\in\mathbb{R}_{+}$ such that $x_{v}(t)=x,t<T_{1}(v)$ (simply take $v(x,t)=0$ if $x=0$).

Proof.

Step 1. For any $0<v_{1}<v_{2}\leq v_{0}$, we prove that $y_{v_{1}}(t)<y_{v_{2}}(t),\forall t\in(0,T_{1}(v_{1})]$. Otherwise suppose $y_{v_{1}}(t)=y_{v_{2}}(t)$ for some $t\in(0,T_{1}(v_{1})]$. If $t\leq T(v_{1})$, as in the proof of Lemma 3.1 we have

[TABLE]

which is impossible since $G(z)+v_{1}^{2}<G(z)+v_{2}^{2}$. If $t\in(T(v_{1}),T(v_{2})]$, then $y_{v_{2}}(t)>y_{v_{2}}(T(v_{1}))>y_{v_{1}}(T(v_{1}))>y_{v_{1}}(t)$, which is contradictory to our assumption. If $t\in(T(v_{2}),T_{1}(v_{1})]$, we have

[TABLE]

which contradicts to Lemma 3.2.

Step 2. For any v_{0}\leq v_{1}<v_{2},t\in\big{(}0,\int_{0}^{+\infty}\frac{dz}{\sqrt{G(z)+v^{2}}}\big{]}, we have $y_{v_{1}}(t)<y_{v_{2}}(t)$, which can be proved as in Step 1.

Step 3. For any $0<v_{1}<v_{2}\leq v_{0}$, we prove that $x_{v_{1}}(t)<x_{v_{2}}(t),\forall t\in[0,T_{1}(v_{1})]$. Otherwise suppose $t=\sup\{t:x_{v_{1}}(t)=x_{v_{2}}(t),t\leq T_{1}(v_{1})\}$, where supreme is attained by the continuity of $x_{v_{1}}(.)$ and $x_{v_{2}}(.)$. To show the contradiction, we prove that $\frac{d}{dt}(x_{v_{2}}(t)-x_{v_{1}}(t))<0$, in which case these two curves have to intersect after time $t$ since $x_{v_{2}}$ decreases to [math] at time $T_{1}(v_{2})>T_{1}(v_{1})$.

If $t\geq T(v_{1})$, we have

[TABLE]

Since we proved $y_{v_{1}}(t)<y_{v_{2}}(t)$, the derivative $\frac{d}{dt}y_{v_{2}}(t)$ must be negative, and hence

[TABLE]

Combining (3.9) and $\frac{d}{dt}y_{v_{i}}(t)=-\sqrt{G(y_{v_{i}}(t))+v_{i}^{2}},i=1,2$ , we obtain

[TABLE]

Because of (3.9) and the fact that $y_{v_{2}}(t)>y_{v_{1}}(t)$, we deduce that $\frac{d}{dt}(x_{v_{2}}(t)-x_{v_{1}}(t))<0$ is equivalent to $\sqrt{G(y_{v_{2}}(t))+v_{2}^{2}}-\frac{1}{2}y_{v_{2}}(t)^{2}-2\eta y_{v_{2}}(t)+1>0$, which is true since

[TABLE]

If $t<T(v_{1})$, by the same reasoning we have

[TABLE]

and also

[TABLE]

Accordingly, it suffices to show that \bigg{(}\sqrt{G(y_{v_{2}}(t))+v_{2}^{2}}+\frac{1}{2}y_{v_{2}}(t)^{2}+2\eta y_{v_{2}}(t)-1\bigg{)}<0, which is equivalent to

[TABLE]

Taking square of (3.10) , we obtain the equivalent inequality $v_{2}^{2}+4\eta y_{v_{2}}(t)-1<0.$ Since $y_{v_{2}}(t)\leq y(v_{2})$, we conclude our claim by the following computation

[TABLE]

Step 4. For any v_{0}\leq v_{1}<v_{2},t\in\big{(}0,\int_{0}^{+\infty}\frac{dz}{\sqrt{G(z)+v^{2}}}\big{]}, we have $x_{v_{1}}(t)<x_{v_{2}}(t)$, which can be proved as in Step 3.

Step 5. Until now we have shown that the stopped curves $\{x_{v}(t):0\leq t<T_{1}(v)\}$ do not intersect, and it remains to prove that for any $(x,t)\in\mathbb{R}_{+}\times\mathbb{R}_{+}$, there exists a $v(x,t)\in\mathbb{R}_{+}$ such that $x_{v}(t)=x,t<T_{1}(v)$. Note that according to (3.4), for any fixed $t$, the couple $(y_{v}(t),\frac{d}{dt}y_{v}(t))$ is continuous with respect to the initial velocity $v$, and thus the mapping $v\mapsto x_{v}(t)$ is also continuous.

First suppose $x<x_{v_{0}}(t)$ and $t\leq T_{1}(0+)$. As a result of $\lim\limits_{v\to 0}x_{v}(t)=0,\lim\limits_{v\to v_{0}}x_{v}(t)=x_{v_{0}}(t)$ and the continuity of $v\mapsto x_{v}(t)$, we know that there must exist some $v\in(0,v_{0})$ such that $x_{v}(t)=x$. The equality $t<T_{1}(v)$ simply follows from the inequality $t\leq T_{1}(0+)<T_{1}(v)$.

Suppose $x<x_{v_{0}}(t)$ and $t>T_{1}(0+)$. Since $T_{1}(v)$ increases to $+\infty$ as $v$ increases to $v_{0}$, we know that there exists a unique $v^{\prime}\in(0,v_{0})$ such that $t=T_{1}(v^{\prime})$, which also implies $x_{v^{\prime}}(t)=0$. According to the continuity of $v^{\prime}\mapsto x_{v^{\prime}}(t)$, and the fact that $\lim\limits_{v\to v_{0}}x_{v}(t)=x_{v_{0}}(t)$, we know there must exist a $v>v^{\prime}$ such that $x_{v}(t)=x$, and $t=T_{1}(v^{\prime})<T_{1}(v)$.

In the end suppose $x>x_{v_{0}}(t)$. Because the mapping $v\mapsto\int_{0}^{+\infty}\frac{dz}{\sqrt{G(z)+{v}^{2}}}$ is decreasing from $+\infty$ to [math] over the interval $(v_{0},+\infty)$, there exists a unique $v^{\prime}>v_{0}$ such that $\int_{0}^{+\infty}\frac{dz}{\sqrt{G(z)+{v^{\prime}}^{2}}}=t$, which also implies $x_{v^{\prime}}(t)=+\infty$. Again by the continuity of $v\mapsto x_{v}(t)$ and the fact that $\lim\limits_{v\to v_{0}}x_{v}(t)=x_{v_{0}}(t)<x$, there exists a $v>v_{0}$ such that $x_{v}(t)=x$. ∎

Proposition 3.2**.**

Suppose $\eta<\frac{1}{2}$. Then there exists a unique solution to (3.4) for any $T>0$ if $|2\bar{\theta}-1|\geq 1-{\eta}^{2}-\eta\sqrt{{\eta}^{2}+2}$, and the number of solutions to (3.4) can be arbitrarily large if $|2\bar{\theta}-1|<1-{\eta}^{2}-\eta\sqrt{{\eta}^{2}+2}$ and $T$ is large enough. In particular, the number of solutions with boundary condition $2\bar{\theta}-1=0$ is given by

[TABLE]

Proof.

Recalling $v_{0}=\sqrt{-G(\sqrt{{\eta}^{2}+2}-3\eta)}$, we first prove that $x_{v_{0}}(t)$ is increasing with respect to $t$ and $\lim\limits_{t\to+\infty}x_{v_{0}}(t)=1-{\eta}^{2}-\eta\sqrt{{\eta}^{2}+2}$.

Taking derivative of the following equation,

[TABLE]

we get $\frac{d}{dt}x_{v_{0}}(t)=(y_{v_{0}}(t)+2\eta)\frac{d}{dt}y_{v_{0}}(t)+\frac{1}{2}G^{{}^{\prime}}(y_{v_{0}}(t))$. Therefore $x_{v_{0}}(t)$ is increasing is equivalent to

[TABLE]

Since both sides of (3.11) are positive, it is enough to show that

[TABLE]

Plugging in the equality $\frac{d}{dt}y_{v_{0}}(t)=\sqrt{G(y_{v_{0}}(t))+v_{0}^{2}}$ and the formula of $G$, the inequality becomes

[TABLE]

Now we finish proving $x_{v_{0}}(t)$ is increasing by the following equality,

[TABLE]

Recall Lemma 3.1, $y_{v_{0}}(t)$ is given by the equation

[TABLE]

Combining the equality proved in Lemma 3.2 that $\int_{0}^{\sqrt{{\eta}^{2}+2}-3\eta}\frac{dz}{\sqrt{G(z)+v_{0}^{2}}}=+\infty$, we conclude that $\lim\limits_{t\to+\infty}y_{v_{0}}(t)=\sqrt{{\eta}^{2}+2}-3\eta.$ Also, according to (3.6), we get that

[TABLE]

Therefore by (3.5), we conclude the second claim

[TABLE]

It can be seen that the curves $\{x_{v}(t):t\geq 0,v\geq v_{0}\}$ never cross each other, and that $x_{v}(t)<1-{\eta}^{2}-\eta\sqrt{{\eta}^{2}+2}$ for any $t>0$ if $v<v_{0}$. Therefore according to Lemma 3.3, if $|2\bar{\theta}-1|\geq 1-{\eta}^{2}-\eta\sqrt{{\eta}^{2}+2}$, there exists only one $v\geq v_{0}$ such that $x_{v}(T)=2\bar{\theta}-1$.

Now suppose that $0<2\bar{\theta}-1<1-{\eta}^{2}-\eta\sqrt{{\eta}^{2}+2}$. For each $v\in(0,v_{0})$, define

[TABLE]

As a result of Lemma 3.3, $M(v)$ is actually an increasing function, and there exists a unique $\bar{v}\in(0,v_{0})$ such that $M(\bar{v})=2\bar{\theta}-1$. Also for any $v\in[\bar{v},v_{0})$, we can define $t(v)$ as the unique $t$ satisfying $x_{v}(t)=2\bar{\theta}-1,t<T_{1}(v),$ which is also an increasing function of $v$. Then $(x_{v}(.),y_{v}(.))$ is a solution of (3.3) with time horizon $T=t(v)$. Since the period of $x_{v}(.)$ is $4T(v)$, and $\lim\limits_{v\to v_{0}}t(v)=+\infty$, for each $k\in\mathbb{N}$ we know that if $T>t(\bar{v})+4kT(\bar{v})$, there must exist some $v^{\prime}\in[\bar{v},v_{0})$ such that $T=t(v^{\prime})+4kT(v^{\prime})$. Therefore we conclude that the number of solutions to (3.3) with time horizon $T$ is greater than

[TABLE]

which can be arbitrarily large if $T$ is large enough.

In the end, we consider the number of solutions for the terminal condition $2\bar{\theta}-1=0$. We have already shown that $T_{k}(v)$ is the time when $x_{v}(t)$ attains zero. According to Lemma 3.2, the functions $T_{k}(v)$ are increasing with respect to $v$ for each $k\in\mathbb{N}$ and $\lim\limits_{v\to v_{0}}T_{k}(v)=+\infty$. Since $x_{-v}(t)=-x_{v}(t)$, and $v=0$ is always a solution, the number of solutions is just

[TABLE]

∎

4. The Master Equation

Letting $Y(t,\theta)=U(t,1,\theta)-U(t,0,\theta)$, $x=2\theta-1$, and time reverse the master equation (ME), we obtain the equation

[TABLE]

with the boundary condition $Y(0,x)=0,\forall x\in[-1,1]$.

Since the equation has the form of a scalar conservation law, there exists a unique entropy solution. By the method of characteristics, we directly construct a piecewise $\mathcal{C}^{1}$ solution to (4.1) and then check it is entropic.

Rewriting (4.1) as

[TABLE]

and letting $y(t)=Y(t,x(t)),\frac{d}{dt}x=2\eta x-y+x|y|$, we obtain the characteristic curve of (4.1)

[TABLE]

whose solution is given explicitly in Lemma 3.1. If $\eta\geq\frac{1}{2}$, the solution given by characteristic curves is smooth everywhere. If $\eta<\frac{1}{2}$, the shock curve is taken to be $\gamma(t)=0,t\in\mathbb{R}_{+}.$ See our illustration in Figure 1.

Proposition 4.1**.**

The function $Y(x,t):=y_{v(x,t)}(t)$ is the entropy solution of (4.1) with shock curve $\gamma(t)=0,t>T_{1}(0+)$, where $v(x,t)\in\mathbb{R}$ is defined in Lemma 3.3.

Proof.

It is clear that the function $Y(x,t)$ is $\mathcal{C}^{1}$ outside the shock curve, and we only need to check the $Rankine$-$Hugoniot\ condition$ and the $Lax\ condition$ (see [6, Proposition 3]). Define

[TABLE]

If $t>T_{1}(0+)$, there exists a $v>0$ such that $t=T_{1}(v)$ since $v\mapsto T_{1}(v)$ is increasing to $+\infty$ as $v$ increases to $v_{0}$. Also it can be seen that $\lim\limits_{x\downarrow 0}v(x,t)=v$. According to the discussion above Lemma 3.3, we conclude that $Y_{+}(t)=y_{v}(t)=v=\lim\limits_{x\downarrow 0}v(x,t)$, and similarly $Y_{-}(t)=-\lim\limits_{x\downarrow 0}v(x,t)$. If $t\leq T_{1}(0+)$, the mapping $v\mapsto x_{v}(t)$ is continuous and strictly increasing, which is zero at $v=0$. Therefore $\lim\limits_{x\downarrow 0}v(x,t)=0$, and $Y_{+}(t)=Y_{-}(t)=0$. In summary, we have

[TABLE]

Taking $\mathfrak{g}(x,Y)=2\eta xY+\frac{xY|Y|}{2}-\frac{Y^{2}}{2}-\frac{x^{2}}{2}$, we have

[TABLE]

which verifies the $Rankine$-$Hugoniot\ condition$.

For any $c$ strictly between $Y_{-}(t)$ and $Y_{+}(t)$, $t>T_{1}(0+)$, we have

[TABLE]

[TABLE]

and therefore

[TABLE]

which verifies the $Lax\ condition$. ∎

Remark 4.1**.**

It is easily seen that the entropy solution of (4.1) corresponds to a solution of (ME).

Remark 4.2**.**

By Lemma 3.3, we know that for any $\bar{\theta}\in[0,1]$, there exists a unique $v^{{}^{\prime}}$ such that $x_{v^{{}^{\prime}}}(T)=2\bar{\theta}-1,T<T_{1}(v^{{}^{\prime}})$. Then $(x_{v^{{}^{\prime}}}(T-t),y_{v^{{}^{\prime}}}(T-t))$ solves (3.1), which is the mean field equilibrium induced the entropy solution.

5. $N+1$-player game and the selection of Equilibrium

In this section, we consider the $N+1$-player game and always assume $\eta=0$. Since the model we are considering is invariant under permutation, it can be easily seen that

[TABLE]

and therefore we only need to consider the HJB systems for $V^{N+1}(t,1,\theta)$:

[TABLE]

where the optimal control policy is

[TABLE]

As a result of the local Lipschitz continuity of the HJB equation (5.1), the system can be uniquely solved with terminal condition $V^{N+1}(T,0,\theta)=0$, which provides us the unique Nash Equilibrium of the game. Supposing that the representative player is applying the zero control while the other players are taking the optimal policy, then by the definition of Nash Equilibrium we conclude that

[TABLE]

Now we prove that if the representative player agrees with the majority, then he will keep his state by taking the zero control.

Proposition 5.1**.**

Taking

[TABLE]

for any $\theta\in\{0,\frac{1}{N},\dotso,1\}$ we have

[TABLE]

Proof.

We only prove the first inequality of (5.2) for even $N$, and the rest can be proved similarly. As a result of $Y^{N+1}(t,\frac{1}{2})=0$, it is enough for us to show it for $\theta\geq\frac{1}{2}+\frac{1}{N}$. Take

[TABLE]

According to (5.1), we obtain

[TABLE]

and

[TABLE]

By our terminal condition $V^{N+1}(T,1,\theta)=0$, it is easy to see that $Y^{N+1}(T,\theta)=W^{N+1}(T,\theta)=0$, and both $\frac{d}{dt}Y^{N+1}(T,\theta),\frac{d}{dt}W^{N+1}(T,1-\theta)$ are negative if $\theta>\frac{1}{2}.$ And therefore by the continuity of $V^{N+1}(t,1,\theta)$, there exists a small positive $\epsilon>0$ such that $Y^{N+1}(t,\theta),W^{N+1}(t,1-\theta)$ are positive during the time interval $[T-\epsilon,T)$. Define

[TABLE]

We finish the argument by showing that $Y^{N+1}(t,\theta)$ and $W^{N+1}(t,1-\theta)$ are both positive for $t\in[s,T-\epsilon],\theta>\frac{1}{2}$, which implies $s$ has to be $-\infty$. By the definition of $s$, we have $Y^{N+1}(t,\theta)=-Y^{N+1}(t,1-\theta)\geq 0,W^{N+1}(t,1-\theta)\geq 0$ if $t\in[s,T-\epsilon)$, $\theta>\frac{1}{2}$, and therefore we obtain the following inequality from (5.3),

[TABLE]

Since $V^{N+1}(t,1,\theta)\leq T$, we get that $|Y^{N+1}(t,\theta)|\leq 2T$, $|W^{N+1}(t,\theta)|\leq 2T$ for any $\theta\in\{0,\frac{1}{N},\dotso,1\}$. Therefore $Y^{N+1}(t,\theta)$ is bounded below by the solution of

[TABLE]

which is always positive. Similarly, for $t\in[s,T-\epsilon],\theta>\frac{1}{2}$, we obtain the inequality from (5.4)

[TABLE]

which implies $W^{N+1}(t,1-\theta)>0$ for $t\in[s,T-\epsilon]$. ∎

Remark 5.1**.**

Recall that $\mathbf{Z}(t)$ is the state of the $N+1$ players at time $t$ when agents play the Nash equilibrium given by (HJB). Denote by ${\theta}^{N+1}(t)$ the fraction of players at state [math], i.e.,

[TABLE]

and let $U$ be the solution of (ME) corresponding to the entropy solution of (4.1). According to Proposition 5.1, ${\theta}^{N+1}(t)$ will always stay on one side of $\frac{1}{2}$ if ${\theta}^{N+1}(0)\not=\frac{1}{2}$. In combination with the fact that $U(t,i,\theta)$ is smooth outside the curve $\bar{\gamma}(t)=\frac{1}{2}$, it can be easily seen that $V^{N+1}(t,1,\theta)$ converges to $U(t,1,\theta)$ if $\theta\not=\frac{1}{2}$ (see e.g. [6, Theorem 8] ).

Let $(\xi_{j})_{j\in\mathbb{N}}$ be the i.i.d initial datum of $Z_{j}$ such that $\mathbb{P}[\xi_{j}=0]=\bar{\theta}\not=\frac{1}{2},\mathbb{P}[\xi_{j}=1]=1-\bar{\theta}.$ Denote by $\tilde{Z}_{j}$ the i.i.d process in which players choose the optimal control $\tilde{\alpha}(t,i):=(U(t,i,\theta(t))-U(t,1-i,\theta(t)))_{+}$, where $U$ is the corresponding entropy solution of (ME). Also, we can prove the propagation of chaos property by using the technique developed in [5] and [6].

6. Conclusion

When $\eta>1/2$, the N-player game converges to the mean field game following the analysis of [1] and [5]. Here we considered the case when $\eta=0$ and showed that the N-player game value functions converge to the entropic mean-field game solution and verified in this case the conjecture of [7].

When $\eta\in(0,\frac{1}{2})$, it is always possible for players to jump to the other state. Therefore $\theta^{N+1}(t)$ may not always stay on one side of $\frac{1}{2}$, and when we use Itô’s formula to the entropy solution $U$, there would be extra jump terms. Subsequently our strategy does not work when $\eta\in(0,1/2)$, and new techniques are needed. We leave this as an open problem.

When $\bar{\theta}=1/2$, it is expected that the N player limit will charge the two solutions we obtain with equal probability (as in [7]), which is numerically justified by the Figure 3 of [10]. Hence in that case the $N$-player empirical distribution will not converge to the stable fixed points of the MFG map (in the language of [7]) unlike what is claimed in the conjecture.