Conservation laws arising in the study of forward-forward Mean-Field Games

TL;DR

This paper explores the connection between forward-forward Mean Field Game models and hyperbolic conservation laws, analyzing their mathematical properties and long-term behavior to advance understanding of stationary MFGs.

Contribution

It establishes a novel link between forward-forward MFGs and hyperbolic conservation laws, and studies existence and long-time dynamics of solutions.

Findings

Link established between forward-forward MFGs and hyperbolic conservation laws

Existence of solutions for the models proven

Long-time behavior of solutions analyzed

Abstract

We consider forward-forward Mean Field Game (MFG) models that arise in numerical approximations of stationary MFGs. First, we establish a link between these models and a class of hyperbolic conservation laws as well as certain nonlinear wave equations. Second, we investigate existence and long-time behavior of solutions for such models.

Equations

$$\{\begin{array}{l}-u_t+H(x,Du)=\epsilon \Delta u+g[m]{\mathbb{T}}^d\times [0,T]\\ m_t-\mathrm{div}(D_{p}H(x,Du(x)m))=\epsilon \Delta m{\mathbb{T}}^d\times [0,T].\end{array}$$

$$\{\begin{array}{l}u(x,T)=u_T(x)\\ m(x,0)=m_0(x).\end{array}$$

$$\{\begin{array}{l}{u}_t+H(x,Du)=\epsilon \Delta u+g[m]\\ m_t-\mathrm{div}(D_{p}H(x,Du(x))m)=\epsilon \Delta m.\end{array}$$

$$\{\begin{array}{l}u(x,0)=u_0(x)\\ m(x,0)=m_0(x)\end{array}$$

$$\{\begin{array}{l}{u}_t+H(u_x)=g(m),\\ m_t-(m{H}^{\prime }(u_x){)}_x=0.\end{array}$$

$$u_{tt}-u_{xx}=0.$$

$$u(x,t)=u_0(x-t)+\frac{1}{2}{\int }_{x-t}^{x+t}g(m_0(s))ds,$$

$$m(x,t)=m_0(x-t).$$

$$u_{tt}-(1+u_x^2)u_{xx}=0.$$

$$\{\begin{array}{l}{v}_t-w_x=0\\ w_t-\sigma (v{)}_x=0\end{array}$$

$$w=u_t,v=u_x,and\sigma (z)=z+\frac{z^3}{3}.$$

$$\{\begin{array}{l}{u}_t+u_x^2/2=m^2/2,\\ m_t-(m{u}_x{)}_x=0.\end{array}$$

$$\{\begin{array}{l}u(x,0)=u_0(x)\\ m(x,0)=m_0(x)>0.\end{array}$$

$$\{\begin{array}{l}{v}_t+{\left(v^2/2-m^2/2\right)}_x=0,\\ m_t-(mv{)}_x=0.\end{array}$$

$$F(v,m)=(v^2/2-m^2/2,-vm)m>0,v\in \mathbb{R}.$$

$$DF(v,m)=\left[\begin{array}{cc}v& -m\\ -m& -v\end{array}\right].$$

$${\lambda }_1=-\sqrt{v^2+m^2}and{\lambda }_2=\sqrt{v^2+m^2},$$

$$r_1=\left[\begin{array}{c}-v+\sqrt{v^2+m^2}\\ m\end{array}\right]and{r}_2=\left[\begin{array}{c}v+\sqrt{v^2+m^2}\\ -m\end{array}\right].$$

$$\nabla {\lambda }_1\cdot r_1=\frac{-m^2+v\left(v-\sqrt{v^2+m^2}\right)}{m\sqrt{v^2+m^2}}$$

$$\nabla {\lambda }_2\cdot r_2=\frac{m^2-v\left(v+\sqrt{v^2+m^2}\right)}{m\sqrt{v^2+m^2}}.$$

$$\nabla {\lambda }_i\cdot r_i=0⟺m^2-v\left(v+\sqrt{v^2+m^2}\right)=0i=1,2.$$

$$\mathcal{S}:=\{(v,m):m^2=3v^2,m>0\}.$$

$$w_1(v,m)=\sqrt{(m^2+v^2{)}^3}-v^3+3v{m}^2$$

$$w_2(v,m)=\sqrt{(m^2+v^2{)}^3}+v^3-3v{m}^2,$$

$$(v+\sqrt{m^2+v^2}){\partial }_v{w}_1-m{\partial }_m{w}_1=0.$$

$$(-v+\sqrt{m^2+v^2}){\partial }_v{w}_1+m{\partial }_m{w}_1=0.$$

$$\{\begin{array}{l}{v}_t+(v^2/2-m^2/2{)}_x=\epsilon v_{xx},\\ m_t-(mv{)}_x=\epsilon m_{xx}.\end{array}$$

$$\underset{\mathbb{T}}{\int }v(x,0)dx=0,\underset{\mathbb{T}}{\int }m(x,0)dx=1,$$

$$\underset{t\to \infty }{\lim }\underset{\mathbb{T}}{\int }|v(x,t)|dx=0,\underset{t\to \infty }{\lim }\underset{\mathbb{T}}{\int }|m(x,t)-1|dx=0.$$

Full text

1 Introduction

A few years ago, Lasry and Lions ll3 and Caines, Huang and Malhame Caines1 independently introduced the Mean-Field Game (MFG) framework. These games model competitive interactions in a population of agents with a dynamics given by an optimal control problem. A typical MFG is determined by the system

[TABLE]

Here, $\mathbb{T}^{d}$ is the $d$-dimensional torus and the Hamiltonian, $H$, the coupling, $g$, and the terminal time, $T>0$, are prescribed. The first equation in (1) is a Hamilton-Jacobi equation. This equation states the optimality of the value function, $u$, associated with the control problem. The second equation is the Fokker-Planck equation that determines the evolution of the density of the agents, $m$. $\varepsilon\geqslant 0$ is a viscosity parameter. If $\varepsilon=0$, we refer to (1) as a first-order MFG. Otherwise, we refer to (1) as a parabolic MFG. System (1) is typically complemented with initial-terminal conditions:

[TABLE]

Extensive research has been conducted in the study of MFGs. For the parabolic problem, strong and weak solutions were, respectively, examined in GPim2 ; GPim1 ; ll2 and ll2 ; porretta2 . The stationary problem for the parabolic case has also generated great interest - several results on the existence of classical and weak solutions were obtained in GMit ; GP ; GPat ; GPatVrt . The uniqueness of a solution in all these cases relies on the monotonicity of $g$.

Here, we consider a related problem, forward-forward MFG, that is derived from (1) by reversing the time in the Hamilton-Jacobi equation. Accordingly, we consider the system

[TABLE]

Because of the time reversal, we prescribe initial-initial conditions:

[TABLE]

for (3). Forward-forward models were first introduced in DY to numerically approximate solutions of stationary MFGs. Before our contributions GomNurSed16 , no rigorous results on the long-time convergence of forward-forward MFGs had been proven. Additionally, the forward-forward problem is interesting on its own right as a learning game. In a standard MFG, agents follow optimal trajectories of a terminal-value optimal control problem. For the forward-forward problem, only initial data is given. Thus, only past optimal trajectories are relevant to the density evolution. Accordingly, the density evolution feeds on past information of the density.

This paper complements the results in GomNurSed16 by examining several cases that can be studied explicitly. In Section 2, we consider linear Hamiltonians and show that the wave equation is a special case of the forward-forward model. In Section 3, we study quadratic forward-forward MFGs using elementary conservation law techniques. In particular, we compute Riemann invariants and characterize invariant domains. We end the paper by recalling the main result from GomNurSed16 on the convergence of forward-forward MFGs.

The study of forward-forward MFG presents substantial challenges even in dimension one which we consider here. The first-order forward-forward problem can be rewritten as a nonlinear wave equation that inherits the non-linearity of the Hamiltonian. For quadratic Hamiltonian, the system reduces to elastodynamics equation. In general, the forward-forward problem can be rewritten, formally, as a system of one-dimensional conservation law. This reformulation allows us to use methods and ideas from the theory of conservation laws such as hyperbolicity, genuinely nonlinearity, Riemann invariants and invariant domains.

For the parabolic forward-forward problem, standard techniques yield existence and uniqueness of a solution. Here, we investigate the long-time convergence of this solution to the solution of a stationary MFG.

2 First-order, one-dimensional, forward-forward Mean-Field Games as nonlinear wave equations

In this section, we consider the first-order, one-dimensional, forward-forward MFG:

[TABLE]

To gain insight into the system above, we consider two simple examples: linear and quadratic Hamiltonians. First, we assume that $H$ is linear, that is, $H(p)=p$, and that the coupling, $g$, is smooth invertible with $g^{\prime}\neq 0$. In this case, $u$ satisfies the wave equation:

[TABLE]

Thus, for smooth initial data in $\eqref{eq: ff initial data}$, solutions of (5) are:

[TABLE]

and

[TABLE]

Next, we assume that $H$ is quadratic, $H(p)=p^{2}/2$, and that $g$ is logarithmic, $g(m)=\ln(m)$. Then, after elementary computations, we obtain that $u$ satisfies the nonlinear wave equation

[TABLE]

This nonlinear equation is known in elastodynamics and, in Lagrangian coordinates, it is a system of hyperbolic conservation laws.

[TABLE]

with

[TABLE]

The system (10) falls within a class of conservation laws investigated in DiPerna83 , in the whole space, and in DeStTz00 , in the periodic case.

3 One-dimensional forward-forward Mean-field Games as conservation laws

In this section, we discuss how certain one-dimensional forward-forward MFGs can be written as a system of one-dimensional conservation laws. Furthermore, we analyze latter and compute the corresponding Riemann invariants. For simplicity, we consider the forward-forward problem with quadratic Hamiltonian and a quadratic coupling:

[TABLE]

We complement (11) with initial-initial condition

[TABLE]

We note that the Fokker-Planck equation preserves positivity. As such, the density $m(t,\cdot)$ is positive for all $t>0$. We formally differentiate the first equation with respect to $x$ and then set $v=u_{x}$. As a result, we obtain

[TABLE]

The associated flux function to the system of conservation laws (13) is given by

[TABLE]

Finally, we compute its Jacobian and get

[TABLE]

3.1 Hyperbolicity and Genuine Nonlinearity

A simple computation shows that (15) has two distinct eigenvalues, $\lambda_{1}$ and $\lambda_{2}$, given by

[TABLE]

with respective eigenvectors given by

[TABLE]

From the discussion above, the system of conservation laws in (13) is strictly hyperbolic. Note that

[TABLE]

and

[TABLE]

Observe that

[TABLE]

As a result, (13) is a strictly hyperbolic genuinely nonlinear system outside the set $\mathcal{S}$ given by

[TABLE]

3.2 Riemann invariants and invariant domains

In the following proposition, we provide an explicit expression for Riemann invariants for the system of conservation laws in the quadratic case. As a consequence, we obtain invariant sets for the corresponding problem with viscosity.

Proposition 1

The system of conservation laws (11) has the following Riemann invariants

[TABLE]

and

[TABLE]

corresponding to the eigenvectors $r_{1}$ and $r_{2}$.

Proof

Note that $w_{i}$ is such that $\nabla w_{i}$ is parallel to the eigenvector $r_{i}$. This means that $w_{1}$ solves

[TABLE]

In a similar way, for $w_{2}$,

[TABLE]

$\blacksquare$

Using Riemann invariants, we can identify invariant domains for the viscosity solutions to (13). These are obtained by looking at level curves of $w_{1}$ and $w_{2}$ see Fig. 1.

4 Convergence of one-dimensional, forward-forward, parabolic conservation laws

Here, we consider (1) in dimension 1. As before, by differentiating the first equation with respect to $x$ and setting $v=u_{x}$, we get

[TABLE]

The system (24) has a unique local smooth solution for bounded initial data. Now, we investigate the long-time convergence of the solution. For that, we, additionally, require

[TABLE]

which are natural assumptions from the perspective of periodic MFGs. The following theorem is proven in GomNurSed16 .

Theorem 4.1

If $v,m\in C^{2}({\mathbb{T}}\times(0,+\infty))\cap C({\mathbb{T}}\times[0,+\infty)),\ m>0,$ solve (24) and satisfy (25) then, we have that

[TABLE]