Linear-Quadratic McKean-Vlasov Stochastic Differential Games

TL;DR

This paper studies multi-player stochastic differential games with linear McKean-Vlasov dynamics and quadratic costs, providing a direct solution approach and characterizing Nash equilibria via Riccati and backward stochastic differential equations.

Contribution

It introduces a simple direct method using weak martingale optimality and fixed point arguments to solve mean-field games with linear McKean-Vlasov dynamics.

Findings

Characterization of Nash equilibria through Riccati ODEs and mean-field BSDEs.

Existence and uniqueness conditions for the equilibrium systems.

Application to a toy example demonstrating the theoretical results.

Abstract

We consider a multi-player stochastic differential game with linear McKean-Vlasov dynamics and quadratic cost functional depending on the variance and mean of the state and control actions of the players in open-loop form. Finite and infinite horizon problems with possibly some random coefficients as well as common noise are addressed. We propose a simple direct approach based on weak martingale optimality principle together with a fixed point argument in the space of controls for solving this game problem. The Nash equilibria are characterized in terms of systems of Riccati ordinary differential equations and linear mean-field backward stochastic differential equations: existence and uniqueness conditions are provided for such systems. Finally, we illustrate our results on a toy example.

Equations

$${\mathcal{A}}^i=\left\{{\alpha }_i:\Omega \times [0,T]\to {\mathbb{R}}^{d_i}\text{s.t. }{\alpha }_i\text{is F-adapted and}{\int }_0^T{e}^{-\rho t}\mathbb{E}[|{\alpha }_{i,t}{|}^2]dt<\infty \right\},$$

$$\{\begin{array}{ll}d{X}_t& =b(t,X_t^{\mathbf{\alpha }},\mathbb{E}\left[X_t^{\mathbf{\alpha }}\right],{\mathbf{\alpha }}_t,\mathbb{E}\left[{\mathbf{\alpha }}_t\right])dt+\sigma (t,X_t^{\mathbf{\alpha }},\mathbb{E}\left[X_t^{\mathbf{\alpha }}\right],{\mathbf{\alpha }}_t,\mathbb{E}\left[{\mathbf{\alpha }}_t\right])d{W}_t,0\le t\le T,\\ X_0^{\alpha }& =X_0,\end{array}$$

$$\{\begin{array}{ll}b(t,x,\overline{x},\mathbf{\alpha },\overline{\mathbf{\alpha }})& ={\beta }_t+b_{x,t}x+{\tilde{b}}_{x,t}\overline{x}+{\sum }_{i=1}^n{b}_{i,t}{\alpha }_i+{\tilde{b}}_{i,t}{\overline{\alpha }}_i\\ & ={\beta }_t+b_{x,t}x+{\tilde{b}}_{x,t}\overline{x}+B_t\mathbf{\alpha }+{\tilde{B}}_t\overline{\mathbf{\alpha }}\\ \sigma (t,x,\overline{x},\mathbf{\alpha },\overline{\mathbf{\alpha }})& ={\gamma }_t+{\sigma }_{x,t}x+{\tilde{\sigma }}_{x,t}\overline{x}+{\sum }_{i=1}^n{\sigma }_{i,t}{\alpha }_i+{\tilde{\sigma }}_{i,t}{\overline{\alpha }}_i\\ & ={\gamma }_t+{\sigma }_{x,t}x+{\tilde{\sigma }}_{x,t}\overline{x}+{\Sigma }_t\mathbf{\alpha }+{\tilde{\Sigma }}_t\overline{\mathbf{\alpha }}.\end{array}$$

$$J^i({\alpha }_i,{\alpha }^{-i})$$

$$V^i({\alpha }^{-i})$$

$$\{\begin{array}{ll}{f}^i(t,x,\overline{x},\mathbf{a},\overline{\mathbf{a}})& =(x-\overline{x}{)}^{⊺}{Q}_t^i(x-\overline{x})+{\overline{x}}^{⊺}[Q_t^i+{\tilde{Q}}_t^i]\overline{x}\\ & +{\sum }_{k=1}^n{a}_k^{⊺}{I}_{k,t}^i(x-\overline{x})+{\overline{a}}_k^{⊺}(I_{k,t}^i+{\tilde{I}}_{k,t}^i)\overline{x}\\ & +{\sum }_{k=1}^n(a_k-{\overline{a}}_k{)}^{⊺}{N}_{k,t}^i(a_k-{\overline{a}}_k)+{\overline{a}}_k(N_{k,t}^i+{\tilde{N}}_{k,t}^i){\overline{a}}_k\\ & +{\sum }_{0\le k\ne l\le n}(a_k-{\overline{a}}_k{)}^{⊺}{G}_{k,l,t}^i(a_l-{\overline{a}}_l)+a_k^{⊺}(G_{k,l,t}^i+{\tilde{G}}_{k,l,t}^i)a_l\\ & +2[L_{x,t}^{iT}x+{\sum }_{k=1}^n{L}_{k,t}^{i⊺}{a}_k]\\ g^i(x,\overline{x})& =(x-\overline{x}{)}^{⊺}{P}^i(x-\overline{x})+\overline{x}(P^i+{\tilde{P}}^i)\overline{x}+2r^{i⊺}x.\end{array}$$

$$J^i({\mathbf{\alpha }}^{\ast })\le J^i({\alpha }_i,{\alpha }^{\ast ,-i}),\forall {\alpha }_i\in {\mathcal{A}}^i,\text{mbox}i.e.,J^i({\mathbf{\alpha }}^{\ast })=V^i({\alpha }^{\ast ,-i}).$$

$$B{R}_i:{\mathcal{A}}^{-i}$$

$${\alpha }^{-i}$$

$$\begin{array}{rl}{\alpha }_{i,t}& =-(S_{i,t}^i{)}^{-1}{U}_{i,t}^i(X_t-\mathbb{E}[X_t])-(S_{i,t}^i{)}^{-1}({\xi }_{i,t}^i-{\overline{\xi }}_{i,t}^i)-({\hat{S}}_{i,t}^i{)}^{-1}(V_{i,t}^i\mathbb{E}[X_t]+O_{i,t}^i)\end{array}$$

$$\begin{array}{rl}{L}^{\infty }([0,T],\mathbb{K})& =\left\{\varphi :[0,T]\to \mathbb{K}\text{s.t. ϕ is measurable and}\underset{t\in [0,T]}{\sup }|{\varphi }_t|<\infty \right\}\\ L^2([0,T],\mathbb{K})& =\left\{\varphi :[0,T]\to \mathbb{K}\text{s.t. ϕ is measurable and}{\int }_0^T{e}^{-\rho u}|{\varphi }_t{|}^{2}du<\infty \right\}\\ L_{{\mathcal{F}}_T}^2(\mathbb{K})& =\left\{\varphi :\Omega \to \mathbb{K}\text{s.t. ϕ is FT-measurable and}\mathbb{E}[|\varphi {|}^2]<\infty \right\}\\ {\mathcal{S}}_{\mathbb{F}}^2(\Omega \times [0,T],\mathbb{K})& =\left\{\varphi :\Omega \times [0,T]\to \mathbb{K}\text{s.t. ϕ is F-adapted and}\mathbb{E}[\underset{t\in [0,T]}{\sup }|{\varphi }_t{|}^2]<\infty \right\}\\ L_{\mathbb{F}}^2(\Omega \times [0,T],\mathbb{K})& =\left\{\varphi :\Omega \times [0,T]\to \mathbb{K}\text{s.t. ϕ is F-adapted and}{\int }_0^T{e}^{-\rho u}\mathbb{E}[|{\varphi }_u{|}^2]du<\infty \right\}.\end{array}$$

$$\mathbb{E}\Big{[}\sup_{t\in[0,T]}|X_{t}^{\alpha}|^{2}\Big{]}\leq C_{\alpha}(1+\mathbb{E}(|X_{0}|^{2}))<\infty$$

$$J^i(\mathbf{\alpha })\in \mathbb{R}\text{ for each α∈A},$$

$$\begin{array}{rl}\mathbb{E}[{\mathcal{W}}_0^{({\alpha }_i,\mathbf{\alpha }\star ,-i),i}]& =\mathbb{E}[{\mathcal{S}}_0^{({\alpha }_i,{\mathbf{\alpha }}^{\star ,-i}),i}]\\ & \le \mathbb{E}[{\mathcal{S}}_T^{({\alpha }_i,{\mathbf{\alpha }}^{\star -i}),i}]=J^i({\alpha }_i,{\mathbf{\alpha }}^{\star ,-i}).\end{array}$$

$$\begin{array}{rl}\mathbb{E}[{\mathcal{W}}_0^{({\alpha }_i^{\star },{\mathbf{\alpha }}^{\star ,-i}),i}]& =\mathbb{E}[{\mathcal{S}}_0^{({\alpha }_i^{\star },{\mathbf{\alpha }}^{\star ,-i}),i}]\\ & =\mathbb{E}[{\mathcal{S}}_T^{({\alpha }_i,{\mathbf{\alpha }}^{\star ,-i}),i}]=J^i({\alpha }_i^{\star },{\mathbf{\alpha }}^{\star ,-i}),\end{array}$$

$$\mathbb{E}[{\mathcal{S}}_0^{\tilde{\mathbf{\alpha }},i}]=\mathbb{E}[{\mathcal{W}}_0^{\tilde{\mathbf{\alpha }},i}]=\mathbb{E}[{\mathcal{W}}_0^{{\mathbf{\alpha }}^{\star },i}]=\mathbb{E}[{\mathcal{S}}_T^{{\mathbf{\alpha }}^{\star },i}]=J^i({\mathbf{\alpha }}^{\star })=J^i(\tilde{\mathbf{\alpha }})=\mathbb{E}[{\mathcal{S}}_T^{\tilde{\mathbf{\alpha }},i}].$$

$$\mathbb{E}\big{[}{D^{\alpha,i}_{t}}\big{]}\;=\;\mathbb{E}\Big{[}{-\rho w_{t}^{i}(X_{t}^{\boldsymbol{\alpha}},\mathbb{E}\left[X_{t}^{\boldsymbol{\alpha}}\right])+\frac{d}{dt}\mathbb{E}\left[w_{t}^{i}(X_{t}^{\alpha},\mathbb{E}\left[X_{t}^{\alpha}\right])\right]+f^{i}(t,X^{\boldsymbol{\alpha}}_{t},\mathbb{E}\left[X^{\boldsymbol{\alpha}}_{t}\right],\boldsymbol{\alpha}_{t},\mathbb{E}\left[\boldsymbol{\alpha}_{t}\right])}\Big{]}.$$

$$\{\begin{array}{ll}d{K}_t^i={\dot{K}}_t^{i}dt,& K_T^i=P^i\\ d{\Lambda }_t^i={\dot{\Lambda }}_t^{i}dt,& {\Lambda }_T^i=P^i+{\tilde{P}}^i\\ d{Y}_t^i={\dot{Y}}_t^{i}dt+Z_t^{i}d{W}_t,0\le t\le T,& Y_T^i=r^i\\ d{R}_t^i={\dot{R}}_t^{i}dt,& R_T^i=0,\end{array}$$

$${\mathcal{S}}_t^{\mathbf{\alpha },i}:=e^{-\rho t}{w}_t^i(X_t,\mathbb{E}\left[X_t\right])+{\int }_0^t{e}^{-\rho u}{f}^i(u,X_u^{\mathbf{\alpha }},\mathbb{E}\left[X_u^{\mathbf{\alpha }}\right],{\mathbf{\alpha }}_u,\mathbb{E}\left[{\mathbf{\alpha }}_u\right])\mathrm{d}u$$

$$\begin{array}{rl}\frac{d\mathbb{E}[{\mathcal{S}}_t^{\mathbf{\alpha },i}]}{dt}& =e^{-\rho t}\mathbb{E}[D_t^{\mathbf{\alpha },i}]\\ & =e^{-\rho t}\mathbb{E}[(X_t-\overline{X_t}{)}^{⊺}[{\dot{K}}_t^i+{\Phi }_t^i](X_t-\overline{X_t})+{\overline{X_t}}^{⊺}({\dot{\Lambda }}_t^i+{\Psi }_t^i)\overline{X_t}+2[{\dot{Y}}_t^i+{\Delta }_t^i{]}^{⊺}{X}_t\\ & +{\dot{R}}_t^i-\rho R_t^i+{\overline{\Gamma }}_t^i+{\chi }_t^i({\alpha }_{i,t})],\end{array}$$

$$\begin{array}{rl}{\chi }_t^i({\alpha }_{i,t})& :=({\alpha }_{i,t}-{\overline{\alpha }}_{i,t}{)}^{⊺}{S}_{i,t}^i({\alpha }_i-{\overline{\alpha }}_{i,t})+{\overline{\alpha }}_{i,t}^{⊺}{\hat{S}}_{i,t}^i{\overline{\alpha }}_{i,t}+2[U_{i,t}^i(X_t-\overline{X_t})+V_{i,t}^i\overline{X_t}+O_{i,t}^i+{\xi }_{i,t}^i-{\overline{\xi }}_{i,t}^i{]}^{⊺}{\alpha }_{i,t}\end{array}$$

$$\{\begin{array}{ll}{\Phi }_t^i& =Q_t^i+{\sigma }_{x,t}^{⊺}{K}_t^i{\sigma }_{x,t}+K_t^i{b}_{x,t}+b_{x,t}^{⊺}{K}_t^i-\rho K_t^i\\ {\Psi }_t^i& ={\hat{Q}}_t^i+{\hat{\sigma }}_{x,t}^{⊺}{K}_t^i{\hat{\sigma }}_{x,t}+{\Lambda }_t^i{\hat{b}}_{x,t}+{\hat{b}}_{x,t}^{⊺}{\Lambda }_t^i-\rho {\Lambda }_t^i\\ {\Delta }_t^i& =L_{x,t}^i+b_{x,t}^{⊺}{Y}_t^i+{\tilde{b}}_{x,t}^{⊺}{\overline{Y}}_t^i+{\sigma }_{x,t}^{⊺}{Z}_t^i+{\tilde{\sigma }}_{x,t}^{⊺}{\overline{Z}}_t^i+{\Lambda }_t^i{\overline{\beta }}_t\\ & +{\sigma }_{x,t}^{⊺}{K}_t^i{\gamma }_t+{\tilde{\sigma }}_{x,t}^{⊺}{K}_t^i{\overline{\gamma }}_t+K_t^i({\beta }_t-{\overline{\beta }}_t)-\rho Y_t^i\\ & +{\sum }_{k\ne i}{U}_{k,t}^{i⊺}({\alpha }_{k,t}-{\overline{\alpha }}_{k,t})+V_{k,t}^{i⊺}{\overline{\alpha }}_{k,t}\\ {\Gamma }_t^i& ={\gamma }_t^{⊺}{K}_t^i{\gamma }_t+2{\beta }_t^{⊺}{Y}_t^i+2{\gamma }_t^{⊺}{Z}_t^i\\ & +{\sum }_{k\ne i}({\alpha }_{k,t}-{\overline{\alpha }}_{k,t}{)}^{⊺}{S}_{k,t}^i({\alpha }_{k,t}-{\overline{\alpha }}_{k,t})+{\overline{\alpha }}_{k,t}^{⊺}{\hat{S}}_{k,t}^i{\overline{\alpha }}_{k,t}+2[O_{k,t}^i+{\xi }_{k,t}^i-{\overline{\xi }}_{k,t}^i{]}^{⊺}{\alpha }_{k,t}-\rho R_t^i,\end{array}$$

$$\{\begin{array}{ll}{S}_{k,t}^i& =N_{k,t}^i+{\sigma }_{k,t}^{⊺}{K}_t^i{\sigma }_{k,t}\\ {\hat{S}}_{k,t}^i& ={\hat{N}}_{k,t}^i+{\hat{\sigma }}_{k,t}^{⊺}{K}_t^i{\hat{\sigma }}_{k,t}\\ U_{k,t}^i& =I_{k,t}^i+{\sigma }_{k,t}^{⊺}{K}_t^i{\sigma }_{x,t}+b_{k,t}^{⊺}{K}_t^i\\ V_{k,t}^i& ={\hat{I}}_{k,t}^i+{\hat{\sigma }}_{k,t}^{⊺}{K}_t^i{\hat{\sigma }}_{x,t}+{\hat{b}}_{k,t}^{⊺}{\Lambda }_t^i\\ O_{k,t}^i& ={\overline{L}}_{k,t}^i+{\hat{b}}_{k,t}^{⊺}{\overline{Y}}_t^i+{\hat{\sigma }}_{k,t}^{⊺}{\overline{Z}}_t^i+{\hat{\sigma }}_{k,t}^{⊺}{K}_t^i{\overline{\gamma }}_t\\ & +\frac{1}{2}{\sum }_{k\ne i}({\hat{J}}_{i,k,t}^i+{\hat{J}}_{k,i,t}^{i⊺}){\overline{\alpha }}_{k,t}\\ J_{k,l,t}^i& =G_{k,l,t}^i+{\sigma }_{k,t}^{⊺}{K}_t^i{\sigma }_{l,t}\\ {\hat{J}}_{k,l,t}^i& ={\hat{G}}_{k,l,t}^i+{\hat{\sigma }}_{k,t}^{⊺}{K}_t^i{\hat{\sigma }}_{l,t}\\ {\xi }_{k,t}^i& =L_{k,t}^i+b_{k,t}^{⊺}{Y}_t^i+{\sigma }_{k,t}^{⊺}{Z}_t^i+{\sigma }_{k,t}^{⊺}{K}_t^i{\gamma }_t\\ & +\frac{1}{2}{\sum }_{k\ne i}(J_{i,k,t}^i+J_{k,i,t}^{i⊺}){\alpha }_{k,t}.\end{array}$$

$$\mathbb{E}[{\chi }_t^i({\alpha }_{i,t})]=\mathbb{E}[({\alpha }_{i,t}-{\overline{\alpha }}_{i,t}+{\theta }_t^{i⊺}{\overline{\alpha }}_{i,t}-{\eta }_t^i)S_{i,t}^i({\alpha }_{i,t}-{\overline{\alpha }}_{i,t}+{\theta }_t^{i⊺}{\overline{\alpha }}_{i,t}-{\eta }_t^i)-{\zeta }_t^i]$$

$$\begin{cases}\eta^{i}_{t}&=a^{i,0}_{t}(X_{t},\overline{X_{t}})+\theta^{i\intercal}_{t}a^{i,1}_{t}(\overline{X_{t}})\\ a^{i,0}_{t}(x,\overline{x})&=-\big{(}{S}^{i}_{i,t}\big{)}^{-1}{U}^{i}_{i,t}(x-\overline{x})-\big{(}{S}^{i}_{i,t}\big{)}^{-1}({\xi}^{i}_{i,t}-{\overline{\xi}}^{i}_{i,t})\\ a^{i,1}_{t}(\overline{x})&=-\big{(}\hat{S}_{i,t}^{i}\big{)}^{-1}(V^{i}_{i,t}\overline{x}+O^{i}_{i,t})\\ \zeta^{i}_{t}&=(X_{t}-\overline{X_{t}})^{\intercal}U_{i,t}^{i\intercal}\big{(}S^{i}_{t}\big{)}^{-1}U_{t}^{i}(X_{t}-\overline{X_{t}})+\overline{X_{t}}{V_{i,t}^{i\intercal}\big{(}\hat{S}_{t}^{i}\big{)}^{-1}V_{i,t}^{i}}\overline{X_{t}}\\ &+2(U_{i,t}^{i\intercal}\big{(}S_{t}^{i}\big{)}^{-1}(\xi^{i}_{i,t}-\overline{\xi}^{i}_{i,t})+V_{i,t}^{i}\big{(}\hat{S}^{i}_{i,t}\big{)}^{-1}O^{i}_{i,t})X_{t}\\ &+(\xi^{i}_{i,t}-\overline{\xi}^{i}_{i,t})^{\intercal}\big{(}S_{i,t}^{i}\big{)}^{-1}(\xi^{i}_{i,t}-\overline{\xi}^{i}_{i,t})+O_{i,t}^{i\intercal}\big{(}\hat{S}^{i}_{i,t}\big{)}^{-1}O_{i,t}^{i},\end{cases}$$

$$\begin{array}{rl}\mathbb{E}[D_t^{\mathbf{\alpha },i}]& =\mathbb{E}[(X_t-\overline{X_t}{)}^{⊺}[{\dot{K}}_t^i+{\Phi }_t^{i0}](X_t-\overline{X_t})+{\overline{X_t}}^{⊺}({\dot{\Lambda }}_t^i+{\Psi }_t^{i0})\overline{X_t}+2[{\dot{Y}}_t^i+{\Delta }_t^{i0}{]}^{⊺}{X}_t\\ & +{\dot{R}}_t^i+\overline{{\Gamma }_t^{i0}}\\ & +({\alpha }_{i,t}-{\overline{\alpha }}_{i,t}+{\theta }_t^{i⊺}{\overline{\alpha }}_{i,t}-{\eta }_t^i)S_{i,t}^i({\alpha }_{i,t}-{\overline{\alpha }}_{i,t}+{\theta }_t^{i⊺}{\overline{\alpha }}_{i,t}-{\eta }_t^i)],\end{array}$$

$$\begin{cases}\Phi^{i0}_{t}&=\Phi^{i}_{t}-{U_{i,t}^{i\intercal}\big{(}S^{i}_{i,t}\big{)}^{-1}U_{i,t}^{i}}\\ \Psi^{i0}_{t}&=\Psi^{i}_{t}-{V_{i,t}^{i\intercal}\big{(}\hat{S}_{i,t}^{i}\big{)}^{-1}V_{i,t}^{i}}\\ \Delta^{i0}_{t}&=\Delta^{i}_{t}-U_{i,t}^{i\intercal}\big{(}S_{i,t}^{i}\big{)}^{-1}(\xi^{i}_{i,t}-\overline{\xi}^{i}_{i,t})-V_{i,t}^{i\intercal}\big{(}\hat{S}^{i}_{t}\big{)}^{-1}O^{i}_{i,t}\\ \Gamma_{t}^{i0}&=\Gamma_{t}^{i}-(\xi^{i}_{i,t}-\overline{\xi}^{i}_{i,t})^{\intercal}\big{(}S_{i,t}^{i}\big{)}^{-1}(\xi^{i}_{i,t}-\overline{\xi}^{i}_{t})-O_{i,t}^{i\intercal}\big{(}\hat{S}_{i,t}^{i}\big{)}^{-1}O_{i,t}^{i}.\end{cases}$$

$$\{\begin{array}{ll}d{K}_t^i=-{\Phi }_t^{i0}dt& K_T^i=P^i\\ d{\Lambda }_t^i=-{\Psi }_t^{i0}dt& {\Lambda }_T^i=P^i+{\tilde{P}}^i\\ d{Y}_t^i=-{\Delta }_t^{i0}dt+Z_t^{i}d{W}_t& Y_T^i=r^i\\ d{R}_t^i=-{\Gamma }_t^{i0}dt& R_T^i=0,\end{array}$$

$$\begin{array}{rl}\mathbb{E}[D_t^{\mathbf{\alpha },i}]& =\mathbb{E}[({\alpha }_{i,t}-{\overline{\alpha }}_{i,t}+{\theta }_t^{i⊺}{\overline{\alpha }}_{i,t}-{\eta }_t^i)S_{i,t}^i({\alpha }_{i,t}-{\overline{\alpha }}_{i,t}+{\theta }_t^{i⊺}{\overline{\alpha }}_{i,t}-{\eta }_t^i)]\\ & =\mathbb{E}[({\alpha }_{i,t}-{\overline{\alpha }}_{i,t}-a_t^{i,1}+{\theta }_t^{i⊺}({\overline{\alpha }}_{i,t}-a_t^{i,0}))S_{i,t}^i({\alpha }_{i,t}-{\overline{\alpha }}_{i,t}-a_t^{i,1}+{\theta }_t^{i⊺}({\overline{\alpha }}_{i,t}-a_t^{i,0}))]\end{array}$$

$$\begin{split}\alpha_{i,t}&=a_{t}^{i,0}(X_{t},\mathbb{E}[X_{t}])+a_{t}^{i,1}(\mathbb{E}[X_{t}])\\ &=-\big{(}{S}^{i}_{i,t}\big{)}^{-1}{U}^{i}_{i,t}(X_{t}-\mathbb{E}[X_{t}])-\big{(}{S}^{i}_{i,t}\big{)}^{-1}({\xi}^{i}_{i,t}-{\overline{\xi}}^{i}_{i,t})-\big{(}\hat{S}_{i,t}^{i}\big{)}^{-1}(V^{i}_{i,t}\mathbb{E}[X_{t}]+O^{i}_{i,t})\end{split}$$

$$\begin{array}{rl}{V}^i({\alpha }^{-i})& =\mathbb{E}[{\mathcal{W}}_0^{i,\mathbf{\alpha }}]\\ & =\mathbb{E}[K_0^i.(X_0-{\overline{X}}_0{)}^{\otimes 2}+{\Lambda }_0^i.{\overline{X}}_0^{\otimes 2}+2Y_0^{i⊺}{X}_0+R_0^i].\end{array}$$

Full text

MSC Classification: 49N10, 49L20, 91A13.

Key words: Mean-field SDEs, stochastic differential game, linear-quadratic, open-loop controls, Nash equilibria, weak martingale optimality principle.

1 Introduction

1.1 General introduction-Motivation

The study of large population of interacting individuals (agents, computers, firms) is a central issue in many fields of science, and finds numerous relevant applications in economics/finance (systemic risk with financial entities strongly interconnected), sociology (regulation of a crowd motion, herding behavior, social networks), physics, biology, or electrical engineering (telecommunication). Rationality in the behavior of the population is a natural requirement, especially in social sciences, and is addressed by including individual decisions, where each individual optimizes some criterion, e.g. an investor maximizes her/his wealth, a firm chooses how much to produce outputs (goods, electricity, etc) or post advertising for a large population. The criterion and optimal decision of each individual depend on the others and affect the whole group, and one is then typically looking for an equilibrium among the population where the dynamics of the system evolves endogenously as a consequence of the optimal choices made by each individual. When the number of indistinguishable agents in the population tend to infinity, and by considering cooperation between the agents, we are reduced in the asymptotic formulation to a McKean-Vlasov (McKV) control problem where the dynamics and the cost functional depend upon the law of the stochastic process. This corresponds to a Pareto-optimum where a social planner/influencer decides of the strategies for each individual. The theory of McKV control problems, also called mean-field type control, has generated recent advances in the literature, either by the maximum principle [5], or the dynamic programming approach [14], see also the recent books [3] and [6], and the references therein, and linear quadratic (LQ) models provide an important class of solvable applications studied in many papers, see, e.g., [15], [11], [10], [2].

In this paper, we consider multi-player stochastic differential games for McKean-Vlasov dynamics. This corresponds and is motivated by the competitive interaction of multi-population with a large number of indistinguishable agents. In this context, we are then looking for a Nash equilibrium among the multi-class of populations. Such problem, sometimes refereed to as mean-field-type game, allows to incorporate competition and heterogeneity in the population, and is a natural extension of McKean-Vlasov (or mean-field-type) control by including multiple decision makers. It finds natural applications in engineering, power systems, social sciences and cybersecurity, and has attracted recent attention in the literature, see, e.g., [1], [7], [8], [4]. We focus more specifically on the case of linear McKean-Vlasov dynamics and quadratic cost functional for each player (social planner). Linear Quadratic McKean-Vlasov stochastic differential game has been studied in [9] for a one-dimensional state process, and by restricting to closed-loop control. Here, we consider both finite and infinite horizon problems in a multi-dimensional framework, with random coefficients for the affine terms of the McKean-Vlasov dynamics and random coefficients for the linear terms of the cost functional. Moreover, controls of each player are in open-loop form. Our main contribution is to provide a simple and direct approach based on weak martingale optimality principle developed in [2] for McKean-Vlasov control problem, and that we extend to the stochastic differential game, together with a fixed point argument in the space of open-loop controls, for finding a Nash equilibrium. The key point is to find a suitable ansatz for determining the fixed point corresponding to the Nash equilibria that we characterize explicitly in terms of systems of Riccati ordinary differential equations and linear mean-field backward stochastic differential equations: existence and uniqueness conditions are provided for such systems.

The rest of this paper is organized as follows. We continue Section 1 by formulating the Nash equilibrium problem in the linear quadratic McKean-Vlasov finite horizon framework, and by giving some notations and assumptions. Section 2 presents the verification lemma based on weak submartingale optimality principle for finding a Nash equilibrium, and details each step of the method to compute a Nash equilibrium. We give some extensions in Section 3 to the case of infinite horizon and common noise. Finally, we illustrate our results in Section 4 on some toy example.

1.2 Problem formulation

Let $T>0$ be a finite given horizon. Let $(\Omega,\mathcal{F},\mathbb{F},\mathbb{P})$ be a fixed filtered probability space where $\mathbb{F}=(\mathcal{F}_{t})_{t\in[0,T]}$ is the natural filtration of a real Brownian motion $W$ $=$ $(W_{t})_{t\in[0,T]}$. In this section, for simplicity, we deal with the case of a single real-valued Brownian motion, and the case of multiple Brownian motions will be addressed later in Section 3. We consider a multi-player game with $n$ players, and define the set of admissible controls for each player $i\in\llbracket 1,n\rrbracket$ as:

[TABLE]

where $\rho$ is a nonnegative constant discount factor. We denote by $\boldsymbol{\mathcal{A}}$ $=$ $\mathcal{A}^{1}\times...\times\mathcal{A}^{n}$, and for any $\boldsymbol{\alpha}=(\alpha_{1},...,\alpha_{n})\in\boldsymbol{\mathcal{A}}$, $i$ $\in$ $\llbracket 1,n\rrbracket$, we set $\alpha^{-i}$ $=$ $(\alpha_{1},\ldots,\alpha_{i-1},\alpha_{i+1},\ldots,\alpha_{n})$ $\in$ $\mathcal{A}^{-i}$ $=$ $\mathcal{A}^{1}\times\ldots\times\mathcal{A}^{i-1}\times\mathcal{A}^{i+1}\times\ldots\times\mathcal{A}^{n}$.

Given a square integrable measurable random variable $X_{0}$ and control $\boldsymbol{\alpha}$ $=$ $(\alpha_{1},...,\alpha_{n})$ $\in$ $\boldsymbol{\mathcal{A}}$, we consider the controlled linear mean-field stochastic differential equation in $\mathbb{R}^{d}$:

[TABLE]

where for $t\in[0,T]$, $x,\overline{x}\in\mathbb{R}^{d}$, $a_{i},\overline{a}_{i}\in\mathbb{R}^{d_{i}}$:

[TABLE]

Here all the coefficients are deterministic matrix-valued processes except $\beta$ and $\sigma$ which are vector-valued $\mathbb{F}$-progressively measurable processes.

The goal of each player $i\in\llbracket 1,n\rrbracket$ during the game is to minimize her cost functional over $\alpha_{i}$ $\in$ $\mathcal{A}^{i}$, given the actions $\alpha^{-i}$ of the other players:

[TABLE]

where for each $t\in[0,T]$, $x,\overline{x}\in\mathbb{R}^{d}$, $a_{i},\overline{a}_{i}\in\mathbb{R}^{d_{i}}$, we have set the running cost and terminal cost for each player:

[TABLE]

Here all the coefficients are deterministic matrix-valued processes, except $L_{x}^{i},L_{k}^{i},r^{i}$ which are vector-valued $\mathbb{F}$-progressively measurable processes, and $\intercal$ denotes the transpose of a vector or matrix.

We say that $\boldsymbol{\alpha}^{*}=(\alpha_{1}^{*},...,\alpha_{n}^{*})\in\boldsymbol{\mathcal{A}}$ is a Nash equilibrium if for any $i\in\llbracket 1,n\rrbracket$,

[TABLE]

As it is well-known, the search for a Nash equilibrium can be formulated as a fixed point problem as follows: first, each player $i$ has to compute its best response given the controls of the other players: $\alpha^{\star}_{i}=BR_{i}(\alpha^{-i})$, where $BR_{i}$ is the best response function defined (when it exists) as:

[TABLE]

Then, in order to ensure that $(\alpha^{\star}_{1},...,\alpha^{\star}_{n})$ is a Nash equilibrium, we have to check that this candidate verifies the fixed point equation: $(\alpha^{\star}_{1},...,\alpha^{\star}_{i})=BR(\alpha^{\star}_{1},...,\alpha^{\star}_{i})$ where $BR:=(BR_{1},...BR_{n})$.

The main goal of this paper is to state a general martingale optimality principle for the search of Nash equilibria and to apply it to the linear quadratic case. We first obtain best response functions (or optimal control of each agent conditioned to the control of the others) of each player $i$ of the following form:

[TABLE]

where the coefficients in the r.h.s., defined in (12) and (13), depend on the actions $\alpha^{-i}$ of the other players. We then proceed to a fixed point search for best response function in order to exhibit a Nash equilibrium that is described in Theorem 2.3.

1.3 Notations and Assumptions

Given a normed space $(\mathbb{K},|.|)$, and for $T\in\mathbb{R}^{\star}_{+}$, we set:

[TABLE]

Note that when we will tackle the infinite horizon case we will set $T=\infty$. To make the notations less cluttered, we sometimes denote $X$ $=$ $X^{\boldsymbol{\alpha}}$ when there is no ambiguity. If $C$ and $\tilde{C}$ are coefficients of our model, either in the dynamics or in a cost function, we note: $\hat{C}=C+\tilde{C}$. Given a random variable $Z$ with a first moment, we denote by $\overline{Z}=\mathbb{E}[Z]$. For $M\in\mathbb{R}^{n\times n}$ and $X\in\mathbb{R}^{n}$, we denote by $M.X^{\otimes 2}=X^{\intercal}MX\in\mathbb{R}$. We denote by $\mathbb{S}^{d}$ the set of symmetric $d\times d$ matrices and by $\mathbb{S}^{d}_{+}$ the subset of non-negative symmetric matrices.

Let us now detail here the assumptions on the coefficients.

(H1) The coefficients in the dynamics (3) satisfy:

1. a)

$\beta,\gamma\in L^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{d})$ 2. b)

$b_{x},\tilde{b}_{x},\sigma_{x},\tilde{\sigma}_{x}\in L^{\infty}([0,T],\mathbb{R}^{d\times d})$; $b_{i},\tilde{b}_{i},\sigma_{i},\tilde{\sigma}_{i}\in L^{\infty}([0,T],\mathbb{R}^{d\times d_{i}})$

(H2) The coefficients of the cost functional (4) satisfy:

1. a)

$Q^{i},\tilde{Q}^{i}\in L^{\infty}([0,T],\mathbb{S}^{d}_{+})$, $P^{i},\tilde{P}^{i}\in\mathbb{S}^{d}$, $N^{i}_{k},\tilde{N}_{k}^{i}\in L^{\infty}([0,T],\mathbb{S}^{d_{k}}_{+})$, $I^{i}_{k},\tilde{I}_{k}^{i}\in L^{\infty}([0,T],\mathbb{R}^{d_{k}\times d})$ 2. b)

$L_{x}^{i}\in L^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{d})$, $L_{k}^{i}\in L^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{d_{k}})$, $r^{i}\in L^{2}_{\mathcal{F}_{T}}(\mathbb{R}^{d})$ 3. c)

$\exists\delta>0\;\forall t\in[0,T]$:

$N^{i}_{i,t}\geq\delta\mathbb{I}_{d_{k}}\;\;\;\;\;P^{i}\geq 0\;\;\;\;\;Q^{i}_{t}-I_{i,t}^{i\intercal}(N_{i,t}^{i})^{-1}I_{i,t}^{i}\geq 0$ 4. d)

$\exists\delta>0\;\forall t\in[0,T]$:

$\hat{N}^{i}_{i,t}\geq\delta\mathbb{I}_{d_{k}}\;\;\;\;\;\hat{P}^{i}\geq 0\;\;\;\;\;\hat{Q}^{i}_{t}-\hat{I}_{i,t}^{i\intercal}(\hat{N}_{i,t}^{i})^{-1}\hat{I}_{i,t}^{i}\geq 0.$

Under the above conditions, we easily derive some standard estimates on the mean-field SDE:

By (H1) there exists a unique strong solution to the mean-field SDE (2), which verifies:

[TABLE]

where $C_{\alpha}$ is a constant which depending on $\alpha$ only through $\int_{0}^{T}e^{-\rho t}\mathbb{E}[|\alpha_{t}|^{2}]dt.$ 2. -

By (H2) and (7) we have:

[TABLE]

which means that the optimisation problem is well defined for each player.

2 A Weak submartingale optimality principle to compute a Nash-equilibrium

2.1 A verification Lemma

We first present the lemma on which the method is based.

Lemma 2.1** (Weak submartingale optimality principle).**

*Suppose there exists a couple

$(\boldsymbol{\alpha}^{\star},(\mathcal{W}^{.,i})_{i\in\llbracket 1,n\rrbracket})$, where $\boldsymbol{\alpha}^{\star}\in\boldsymbol{\mathcal{A}}$ and $\mathcal{W}^{.,i}=\{\mathcal{W}^{\boldsymbol{\alpha},i}_{t},t\in[0,T],\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}\}$ is a family of adapted processes indexed by $\boldsymbol{\mathcal{A}}$ for each $i\in\llbracket 1,n\rrbracket$, such that:*

1. (i)

For every $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$, $\mathbb{E}[\mathcal{W}^{\boldsymbol{\alpha},i}_{0}]$ is independent of the control $\alpha_{i}\in\mathcal{A}^{i}$; 2. (ii)

For every $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$, $\mathbb{E}[\mathcal{W}_{T}^{\boldsymbol{\alpha},i}]=\mathbb{E}[g^{i}(X_{T}^{\boldsymbol{\alpha}},\mathbb{P}_{X_{T}^{\boldsymbol{\alpha}}})]$; 3. (iii)

*For every $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$, the map $t\in[0,T]\mapsto\mathbb{E}[\mathcal{S}_{t}^{\boldsymbol{\alpha},i}]$, with *

$\mathcal{S}_{t}^{\boldsymbol{\alpha},i}=e^{-\rho t}\mathcal{W}^{\boldsymbol{\alpha},i}_{t}+\int_{0}^{t}e^{-\rho u}f^{i}(u,X_{u}^{\boldsymbol{\alpha}},\mathbb{P}_{X_{u}^{\boldsymbol{\alpha}}},\boldsymbol{\alpha}_{u},\mathbb{P}_{\boldsymbol{\alpha}_{u}})du$* is well defined and non-decreasing;* 4. (iv)

The map $t\mapsto\mathbb{E}[\mathcal{S}_{t}^{\boldsymbol{\alpha}^{\star},i}]$ is constant for every $t\in[0,T]$;

Then $\boldsymbol{\alpha}^{\star}$ is a Nash equilibrium and $J^{i}(\boldsymbol{\alpha}^{\star})=\mathbb{E}[\mathcal{W}_{0}^{\boldsymbol{\alpha}^{\star},i}]$. Moreover, any other Nash-equilibrium $\tilde{\boldsymbol{\alpha}}$ such that $\mathbb{E}[\mathcal{W}_{0}^{\tilde{\boldsymbol{\alpha}},i}]=\mathbb{E}[\mathcal{W}_{0}^{{\boldsymbol{\alpha}^{\star}},i}]$ and $J^{i}(\tilde{\boldsymbol{\alpha}})=J^{i}({\boldsymbol{\alpha}^{\star}})$ for any $i\in\llbracket 1,n\rrbracket$ satisfies the condition (iv).

Proof.

Let $i\in\llbracket 1,n\rrbracket$ and $\alpha_{i}\in\mathcal{A}^{i}$. From (ii) we have immediately $J^{i}(\boldsymbol{\alpha})=\mathbb{E}\left[\mathcal{S}_{T}^{\boldsymbol{\alpha}}\right]$ for any $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$. We then have:

[TABLE]

Moreover for $\alpha_{i}=\alpha_{i}^{\star}$ we have:

[TABLE]

which proves that $\boldsymbol{\alpha}^{\star}$ is a Nash equilibrium and $J^{i}(\boldsymbol{\alpha}^{\star})=\mathbb{E}[\mathcal{W}_{0}^{\boldsymbol{\alpha}^{\star},i}]$. Finally, let us suppose that $\tilde{\boldsymbol{\alpha}}\in\boldsymbol{\mathcal{A}}$ is another Nash equilibrium such that $\mathbb{E}[\mathcal{W}_{0}^{\tilde{\boldsymbol{\alpha}},i}]=\mathbb{E}[\mathcal{W}_{0}^{{\boldsymbol{\alpha}^{\star}},i}]$ and $J^{i}(\tilde{\boldsymbol{\alpha}})=J^{i}({\boldsymbol{\alpha}^{\star}})$ for any $i\in\llbracket 1,n\rrbracket$. Then, for $i\in\llbracket 1,n\rrbracket$ we have:

[TABLE]

Since $t\mapsto\mathbb{E}[\mathcal{S}_{t}^{\tilde{\boldsymbol{\alpha}},i}]$ is nondecreasing for every $i\in\llbracket 1,n\rrbracket$, this implies that the map is actually constant and (iv) is verified. ∎

2.2 The method and the solution

Let us now apply the optimality principle in Lemma 2.1 in order to find a Nash equilibrium. In the linear-quadratic case the laws of the state and the controls intervene only through their expectations. Thus we will use a simplified optimality principle where $\mathbb{P}$ is simply replaced by $\mathbb{E}$ in conditions (ii) and (iii) of Lemma 2.1. The general procedure is the following:

• Step 1. We guess a candidate for $\mathcal{W}^{\boldsymbol{\alpha},i}$. To do so we suppose that $\mathcal{W}^{\boldsymbol{\alpha},i}_{t}=w_{t}^{i}(X_{t}^{\boldsymbol{\alpha}},\mathbb{E}\left[X_{t}^{\boldsymbol{\alpha}}\right])$ for some parametric adapted random field $\{w_{t}^{i}(x,\overline{x}),t\in[0,T],x,\overline{x}\in\mathbb{R}^{d}\}$ of the form $w_{t}^{i}(x,\overline{x})=K_{t}^{i}.(x-\overline{x})^{\otimes 2}+\Lambda_{t}^{i}.\overline{x}^{\otimes 2}+2Y_{t}^{i\intercal}x+R^{i}_{t}$.
• Step 2. We set $\mathcal{S}^{\boldsymbol{\alpha},i}_{t}=e^{-\rho t}w_{t}^{i}(X_{t}^{\boldsymbol{\alpha}},\mathbb{E}\left[X_{t}^{\boldsymbol{\alpha}}\right])+\int_{0}^{t}e^{-\rho u}f^{i}(u,X_{u}^{\boldsymbol{\alpha}},\mathbb{E}\left[X_{u}^{\boldsymbol{\alpha}}\right],\boldsymbol{\alpha}_{u},\mathbb{E}\left[\boldsymbol{\alpha}_{u}\right])du$ for $i\in\llbracket 1,n\rrbracket$ and $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$ .We then compute \frac{d}{dt}\mathbb{E}\big{[}{\mathcal{S}_{t}^{\boldsymbol{\alpha},i}}\big{]}=e^{-\rho t}\mathbb{E}\big{[}{D^{\alpha,i}_{t}}\big{]} (with Itô’s formula) where the drift $D^{\boldsymbol{\alpha},i}$ takes the form:

[TABLE]

• Step 3. We then constrain the coefficients of the random field so that the conditions of Lemma 2.1 are satisfied. This leads to a system of backward ordinary and stochastic differential equations for the coefficients of $w^{i}$.
• Step 4. At time $t$, given the state and the controls of the other players, we seek the action $\alpha_{i}$ cancelling the drift. We thus obtain the best response function of each player.
• Step 5. We compute the fixed point of the best response functions in order to find an open loop Nash equilibrium $t\mapsto\boldsymbol{\alpha}^{\star}_{t}$.
• Step 6. We check the validity of our computations.

2.2.1 Step 1: guess the random fields

The process $t\mapsto\mathbb{E}\left[w_{t}^{i}(X_{t}^{\boldsymbol{\alpha}},\mathbb{E}\left[X_{t}^{\boldsymbol{\alpha}}\right])\right]$ is meant to be equal to $\mathbb{E}\left[g^{i}(X_{T}^{\boldsymbol{\alpha}},\mathbb{E}\left[X_{T}^{\boldsymbol{\alpha}}\right])\right]$ at time $T$, where $g(x,\overline{x})=P^{i}.(x-\overline{x})^{\otimes 2}+(P^{i}+\tilde{P}^{i}).\overline{x}^{\otimes 2}+r^{i\intercal}x$ with $(P,\tilde{P},r^{i})\in(\mathbb{S}^{d})^{2}\times L^{2}_{\mathcal{F}_{T}}(\mathbb{R}^{d})$. It is then natural to search for a field $w^{i}$ of the form $w_{t}^{i}(x,\overline{x})=K_{t}^{i}.(x-\overline{x})^{\otimes 2}+\Lambda_{t}^{i}.\overline{x}^{\otimes 2}+2Y_{t}^{i\intercal}x+R_{t}^{i}$ with the processes $(K^{i},\Lambda^{i},Y^{i},R^{i})$ in $(L^{\infty}([0,T],\mathbb{S}^{d}_{+})^{2}\times\mathcal{S}^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{d})\times L^{\infty}([0,T],\mathbb{R})$ and solution to:

[TABLE]

where $(\dot{K}^{i},\dot{\Lambda}^{i},\dot{R}^{i})$ are deterministic processes valued in $\mathbb{S}^{d}\times\mathbb{S}^{d}\times\mathbb{R}$ and $(\dot{Y}^{i},Z^{i})$ are adapted processes valued in $\mathbb{R}^{d}$.

2.2.2 Step 2: derive their drifts

For $i\in\llbracket 1,n\rrbracket$, $t\in[0,T]$ and $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$, we set:

[TABLE]

and then compute the drift of the deterministic function $t\mapsto\mathbb{E}[\mathcal{S}^{\boldsymbol{\alpha},i}_{t}]$:

[TABLE]

where we have defined:

[TABLE]

with the following coefficients:

[TABLE]

and

[TABLE]

2.2.3 Step 3: constrain their coefficients

Now that we have computed the drift, we need to constrain the coefficients so that $\mathcal{S}^{\boldsymbol{\alpha},i}$ satisfies the condition of Lemma 2.1. Let us assume for the moment that $S^{i}_{i,t}$ and $\hat{S}^{i}_{i,t}$ are positive definite matrices (this will be ensured by the positive definiteness of $K$). That implies that there exists an invertible matrix $\theta^{i}_{t}$ such that $\theta^{i}_{t}S^{i}_{i,t}\theta_{t}^{i\intercal}=\hat{S}^{i}_{i,t}$ for all $t\in[0,T]$. We can now rewrite the drift as: "a square in $\alpha_{i}$" + "other terms not depending in $\alpha_{i}$". Since we can form the following square:

[TABLE]

with:

[TABLE]

we can then rewrite the drift in the following form:

[TABLE]

where

[TABLE]

We can finally constrain the coefficients. By choosing the coefficients $K^{i},\Gamma^{i},Y^{i}$ and $R^{i}$ so that only the square remains, the drift for each player $i\in\llbracket 1,n\rrbracket$ can be rewritten as a square only (in the next step we will verify that we can indeed choose such coefficients). More precisely we set $K^{i},\Gamma^{i},Y^{i}$ and $R^{i}$ as the solution of:

[TABLE]

and stress the fact that $Y^{i},Z^{i},R^{i}$ depend on $\alpha^{-i}$, which appears in the coefficients $\Delta^{i0}$, and $\Gamma^{i0}$. With such coefficients the drift takes now the form:

[TABLE]

and thus satisfies the nonnegativity constraint: $\mathbb{E}[D^{\boldsymbol{\alpha},i}_{t}]$ $\geq$ [math], for all $t$ $\in$ $[0,T]$, $i$ $\in$ $\llbracket 1,n\rrbracket$, and $\boldsymbol{\alpha}$ $\in$ $\boldsymbol{\mathcal{A}}$.

2.2.4 Step 4: find the best response functions

Proposition 2.2**.**

Assume that for all $i\in\llbracket 1,n\rrbracket$, $(K^{i},\Lambda^{i},Y^{i},Z^{i},R^{i})$ is a solution of (18) given $\alpha^{-i}$ $\in$ ${\cal A}^{-i}$. Then the set of processes

[TABLE]

(depending on $\alpha^{-i}$) where $X$ is the state process with the feedback controls $\boldsymbol{\alpha}=(\alpha_{1},...,\alpha_{n})$, are best-response functions, i.e., $J^{i}(\alpha_{i},\alpha^{-i})$ $=$ $V^{i}(\alpha^{-i})$ for all $i\in\llbracket 1,n\rrbracket$. Moreover we have

[TABLE]

Proof.

We check that the assumptions of Lemma 2.1 are satisfied. Since $\mathcal{W}^{\boldsymbol{\alpha},i}$ is of the form $\mathcal{W}^{\boldsymbol{\alpha},i}_{t}=w_{t}^{i}(X_{t}^{\boldsymbol{\alpha}},\mathbb{E}[X_{t}^{\boldsymbol{\alpha}}])$, condition (i) is verified. The condition (ii) is satisfied thanks to the terminal conditions imposed on the system (18). Since $(K^{i},\Lambda^{i},Y^{i},Z^{i},R^{i})$ is solution to (18), the drift of $t\mapsto\mathbb{E}[\mathcal{S}^{\boldsymbol{\alpha},i}]$ is positive for all $i\in\llbracket 1,n\rrbracket$ and all $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$, which implies condition (iii). Finally, for $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$, we see that $\mathbb{E}[D_{t}^{\boldsymbol{\alpha},i}]\equiv 0$ for $t\in[0,T]$ and $i\in\llbracket 1,n\rrbracket$ if and only if:

[TABLE]

Since $\theta^{i}_{t}$ is invertible, we get $\overline{\alpha}_{i,t}=a^{i,0}_{t}$ by taking the expectation in the above formula. Thus $\mathbb{E}[D_{t}^{\boldsymbol{\alpha},i}]\equiv 0$ for every $i\in\llbracket 1,n\rrbracket$ and $t\in[0,T]$ if and only if $\alpha_{i,t}=\overline{\alpha}_{i,t}+a^{i,1}_{t}=a^{i,1}_{t}+a^{i,0}_{t}$ for every $i\in\llbracket 1,n\rrbracket$ and $t\in[0,T]$. For such controls for the players, the condition (iv) is satisfied. We now check that $\alpha_{i}\in\mathcal{A}^{i}$ for every $i\in\llbracket 1,n\rrbracket$ (i.e. it satisfies the square integrability condition). Since $X$ is solution to a linear Mckean-Vlasov dynamics and satisfies the square integrability condition $\mathbb{E}[\sup_{0\leq t\leq T}|X_{t}|^{2}]<\infty$, it implies that $\alpha_{i}\in L^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{d_{i}})$ since $S^{i}_{i},U^{i}_{i},\hat{S}^{i}_{i},V^{i}_{i}$ are bounded and $(O^{i}_{i},\xi^{i}_{i})\in L^{2}([0,T],\mathbb{R}^{d_{i}})\times L^{2}(\Omega\times[0,T],\mathbb{R}^{d_{i}})$. Therefore $\alpha_{i}\in\mathcal{A}^{i}$ for every $i\in\llbracket 1,n\rrbracket$. ∎

2.2.5 Step 5: search for a fixed point

We now find semi-explicit expressions for the optimal controls of each player. The issue here is the fact that the controls of the other players appear in the best response functions of each player through the vectors $(Y^{1},Z^{1}),...,(Y^{n},Z^{n})$. To solve this fixed point problem, we first rewrite (20) and the backward equations followed by $\boldsymbol{(Y,Z)}=((Y^{1},Z^{1}),...,(Y^{n},Z^{n}))$ in the following way (note that we omit the time dependence of the coefficients to make the notations less cluttered):

[TABLE]

where we define

[TABLE]

Now, the strategy is to propose an ansatz for $t\in[0,T]\mapsto\boldsymbol{Y}_{t}$ in the form:

[TABLE]

where $(\pi,\hat{\pi},\eta)\in L^{\infty}([0,T],\mathbb{R}^{nd\times d})\times L^{\infty}([0,T],\mathbb{R}^{nd\times d})\times\mathcal{S}_{\mathbb{F}}^{2}(\Omega\times[0,T],\mathbb{R}^{nd})$ satisfy:

[TABLE]

By applying Itô’s formula to the ansatz we then obtain:

[TABLE]

By comparing the two Itô’s decompositions of $\boldsymbol{Y}$, we get

[TABLE]

We now substitute the $\boldsymbol{Y}$ by its ansatz in the best response equation (21), and obtain the system:

[TABLE]

To make the next computations slightly less painful we rewrite (25) as

[TABLE]

By injecting (25) into (24) we have:

[TABLE]

Thus we constrain the coefficients $(\pi,\hat{\pi},\psi,\phi)$ of the ansatz of $\boldsymbol{Y}$ to satisfy:

[TABLE]

We now have a feedback form for $\boldsymbol{(Y,Z)}=((Y^{1},Z^{1}),...,(Y^{n},Z^{n}))$. We can inject it in the best response functions $\boldsymbol{\alpha}^{\star}$ in order to obtain the optimal controls in feedback form. We then inject these latter in the state equation in order to obtain an explicit expression of $t\mapsto X_{t}^{\star}$.

2.2.6 Step 6: check the validity

Let us now check the existence and uniqueness of $t\mapsto(K^{i}_{t},\Lambda^{i}_{t},(Y^{i}_{t},Z_{t}^{i}),R^{i}_{t},\pi_{t},\hat{\pi}_{t},(\eta_{t},\phi_{t}))$ where $K^{i}\in L^{\infty}([0,T],\mathbb{S}^{d}_{+})$, $\Lambda^{i}\in L^{\infty}([0,T],\mathcal{S}^{d}_{+})$, $(Y^{i}_{t},Z_{t}^{i})\in\mathcal{S}^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{d})\times L^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{d})$, $R^{i}\in L^{\infty}([0,T],\mathbb{R})$, $\pi,\hat{\pi}\in L^{\infty}([0,T],\mathbb{R}^{nd\times d})$ and $(\eta,\phi)\in\mathcal{S}^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{nd})\times L^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{nd})$, under the assumptions (H1)-(H2). We recall that $t\mapsto(K^{i}_{t},\Lambda^{i}_{t},(Y^{i}_{t},Z_{t}^{i}),R^{i}_{t}$ and $t\mapsto(\pi_{t},\hat{\pi}_{t},(\eta_{t},\phi_{t}))$ are solutions respectively to (18) and (28). Fix $i\in\llbracket 1,n\rrbracket$:

1. (i)

We first consider the coefficients $K^{i}$ which follow Ricatti equations:

[TABLE]

By standard result in control theory (see [16], Ch. 6, Thm 7.2]) under (H1) and (H2) there exists a unique solution $K^{i}\in L^{\infty}([0,T],\mathbb{S}^{d}_{+}).$ 2. (ii)

Given $K^{i}$ let us now consider the $\Lambda^{i}$’s. They also follow Ricatti equations:

[TABLE]

where we define:

[TABLE]

We need the same arguments as for $K^{i}$. The only missing argument to conclude the existence and uniqueness of $\Lambda^{i}$ is: $\hat{Q}^{iK}-(\hat{I}^{iK})^{\intercal}(\hat{N}^{iK})^{-1}\hat{I}^{iK}\geq 0$. As in [2] we can prove with some algebraic calculations that it is implied by the hypothesis $\hat{Q}^{i}-(\hat{I}^{i})^{\intercal}(\hat{N}^{i})^{-1}\hat{I}^{i}\geq 0$ that we made in (H2). 3. (iii)

Given $(K^{i},\Lambda^{i})$ we now consider the equation for $(Y^{i},Z^{i})$ which is a linear mean-field BSDE of the form:

[TABLE]

where the deterministic coefficients $\mathcal{G}^{i}_{t},\hat{\mathcal{G}}^{i},\mathcal{J}^{i},\hat{\mathcal{J}}^{i}\in L^{\infty}([0,T],\mathbb{R}^{d\times d})$ and the stochastic process $\mathcal{V}^{i}\in L^{2}([0,T],\mathbb{R}^{d})$ are defined as:

[TABLE]

By standard results (see Thm. 2.1 in [12]) we obtain that there exists a unique solution $(Y^{i},Z^{i})\in\mathcal{S}_{\mathbb{F}}^{2}(\Omega\times[0,T],\mathbb{R}^{d\times d})\times L_{\mathbb{F}}^{2}(\Omega\times[0,T],\mathbb{R}^{d\times d})$ to (32). 4. (iv)

Given $(K^{i},\Lambda^{i},Y^{i},Z^{i})$ we consider the equation of $R^{i}$ which is a linear ODE whose solution is given by:

[TABLE]

where $h^{i}$ is a deterministic function defined by:

[TABLE]

and the expressions of the coefficients are recalled in (13). 5. (v)

The final step is to verify that the procedure to find a fixed point is valid. More precisely we need to ensure that $t\mapsto(\pi_{t},\hat{\pi}_{t},\eta_{t})$ is well defined. It is difficult to ensure the well posedness of $t\mapsto(\pi_{t},\hat{\pi}_{t})$ for two reasons: first because $\pi$ and $\hat{\pi}$ follow Ricatti equations but are not squared matrices; and second because $\pi$ appears in the equation followed by $\hat{\pi}$. We are not aware of any work addressing this kind of equations in a general setting.

If we suppose $t\mapsto(\pi_{t},\hat{\pi}_{t})$ well defined, then $t\mapsto(\eta_{t},\phi_{t})$ follows a linear mean-field BSDE of the type:

[TABLE]

where the deterministic coefficients $\mathcal{G}^{i}_{t},\hat{\mathcal{G}}^{i},\mathcal{J}^{i},\hat{\mathcal{J}}^{i}\in L^{\infty}([0,T],\mathbb{R}^{nd\times nd})$ and the stochastic process $\mathcal{V}^{i}\in L^{2}([0,T],\mathbb{R}^{nd})$ are defined as:

[TABLE]

Again, by standard results (see Thm. 2.1 in [12]) we obtain that there exists a unique solution $(\eta,\phi)\in\mathcal{S}^{2}(\Omega\times[0,T],\mathbb{R}^{d\times d})\times L_{\mathbb{F}}^{2}(\Omega\times[0,T],\mathbb{R}^{d\times d})$ to (35).

To sum up the arguments previously presented, our main result provides the following characterization of the Nash equilibrium:

Theorem 2.3**.**

Suppose assumptions (H1) and (H2). Suppose also that the system associated with the fixed point search (28) is well defined. Then $\boldsymbol{\alpha}^{\star}=(\alpha_{1},...,\alpha_{n})$ defined by

[TABLE]

where $\boldsymbol{S_{x},S_{y},S_{z},H,\hat{S}_{x},\hat{S}_{y},\hat{S}_{z},\hat{H}}$ are defined in (22), $\boldsymbol{(Y,Z)}$ are in $\mathcal{S}_{\mathbb{F}}^{2}(\Omega\times[0,T],\mathbb{R}^{nd\times d})\times L^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{nd\times d})$ and satisfy (23) and (28), is a Nash equilibrium.

3 Some extensions

3.1 The case of infinite horizon

Let us now tackle the infinite horizon case. The method is similar to the finite-horizon case but some adaptations are needed when dealing with the well posedness of $(K^{i},\Lambda^{i},Y^{i},R^{i})$ and the admissibility of the controls.

We redefine the set of admissible controls for for each player $i\in\llbracket 1,n\rrbracket$ as:

[TABLE]

while the controlled state defined on $\mathbb{R}_{+}$ now follows a dynamics of the form

[TABLE]

where for each $t\in[0,T]$, $x,\overline{x}\in\mathbb{R}^{d}$, $a_{i},\overline{a}_{i}\in\mathbb{R}^{d_{i}}$:

[TABLE]

Notice that now only the coefficients $\beta$ and $\gamma$ are allowed to be stochastic processes. The other linear coefficients are constant matrices.

The goal of each player $i\in\llbracket 1,n\rrbracket$ during the game is still to minimize her cost functional with respect to control $\alpha_{i}$ over $\mathcal{A}^{i}$, and given control $\alpha^{-i}$ of the other players:

[TABLE]

where for each $t\in[0,T]$, $x,\overline{x}\in\mathbb{R}^{d}$, $a_{i},\overline{a}_{i}\in\mathbb{R}^{d_{i}}$ we have set the running cost for each player as:

[TABLE]

Note that the only coefficients that we allow to be time dependent are $L^{i}_{x}$ and $L^{i}_{k}$ for $k\in\llbracket 1,n\rrbracket$ which may be stochastic processes.

3.1.1 Assumptions

We detail below the new assumptions:

(H1’) The coefficients in (40) satisfy:

1. a)

$\beta,\gamma\in L^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{d})$ 2. b)

$b_{x},\tilde{b}_{x},\sigma_{x},\tilde{\sigma}_{x}\in\mathbb{R}^{d\times d}$; $b_{i},\tilde{b}_{i},\sigma_{i},\tilde{\sigma}_{i}\in\mathbb{R}^{d\times d_{i}}$

(H2’) The coefficients of the cost functional (4) satisfy:

1. a)

$Q^{i},\tilde{Q}^{i}\in\mathbb{S}^{d}_{+}$; $N^{i}_{k},\tilde{N}_{k}^{i}\in\mathbb{S}^{d_{k}}_{+}$; $I^{i}_{k},\tilde{I}_{k}^{i}\in\mathbb{R}^{d_{k}\times d}$ 2. b)

$L_{x}^{i}\in L^{2}_{\mathbb{F}}(\Omega\times\mathbb{R}^{\star}_{+},\mathbb{R}^{d})$, $L_{k}^{i}\in L^{2}_{\mathbb{F}}(\Omega\times\mathbb{R}^{\star}_{+},\mathbb{R}^{d_{k}})$ 3. c)

$N^{i}_{k}>0\;\;\;\;\;Q^{i}-I_{i}^{i\intercal}(N_{i}^{i})^{-1}I_{i}^{i}\geq 0$ 4. d)

$\hat{N}^{i}_{k,t}>0\;\;\;\;\;\hat{Q}^{i}-\hat{I}_{i}^{i\intercal}(\hat{N}_{i}^{i})^{-1}\hat{I}_{i}^{i}\geq 0$

(H3’) $\rho>2\left(|b_{x}|+|\tilde{b}_{x}|+8(|\sigma_{x}|^{2}+|\tilde{\sigma}_{x}|^{2})\right)$

As shown below, the new hypothesis (H3’) ensure the well posedness of our problem. Notice first that by (H1’) and classical results, there exists a unique strong solution $X^{\alpha}$ to the SDE (39). Furthermore by (H1’) and (H3’) we obtain by similar arguments as in [2] the following estimate:

[TABLE]

in which $C_{\alpha}$ is a constant depending on $\boldsymbol{\alpha}=(\alpha_{1},...,\alpha_{n})$ only through $\int_{0}^{\infty}e^{-\rho}\mathbb{E}[|\boldsymbol{\alpha}_{u}|^{2}]\mathrm{d}u$. Finally by (H2’) and (43) the minimizing problem (41) is well defined for each player.

3.1.2 A weak submartingale optimality principle on infinite horizon

We now give an easy adaptation of the weak submartingale optimality principle in the case of infinite horizon.

Lemma 3.1** (Weak submartingale optimality principle).**

*Suppose there exists a couple

$(\boldsymbol{\alpha}^{\star},(\mathcal{W}^{.,i})_{i\in\llbracket 1,n\rrbracket})$, where $\boldsymbol{\alpha}^{\star}\in\boldsymbol{\mathcal{A}}$ and $\mathcal{W}^{.,i}=\{\mathcal{W}^{\boldsymbol{\alpha},i}_{t},t\in\mathbb{R}^{\star}_{+},\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}\}$ is a family of adapted processes indexed by $\boldsymbol{\mathcal{A}}$ for each $i\in\llbracket 1,n\rrbracket$, such that:*

1. (i)

For every $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$, $\mathbb{E}[\mathcal{W}^{\boldsymbol{\alpha},i}_{0}]$ is independent of the control $\alpha_{i}\in\mathcal{A}^{i}$; 2. (ii)

For every $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$, $\lim_{t\to\infty}e^{-\rho t}\mathbb{E}\left[\mathcal{W}_{t}^{\boldsymbol{\alpha},i}\right]=0$; 3. (iii)

*For every $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$, the map $t\in\mathbb{R}^{\star}_{+}\mapsto\mathbb{E}\left[\mathcal{S}_{t}^{\boldsymbol{\alpha},i}\right]$, with *

$\mathcal{S}_{t}^{\boldsymbol{\alpha},i}=e^{-\rho t}\mathcal{W}^{\boldsymbol{\alpha},i}_{t}+\int_{0}^{t}e^{-\rho u}f^{i}(u,X_{u}^{\boldsymbol{\alpha}},\mathbb{P}_{X_{u}^{\boldsymbol{\alpha}}},\boldsymbol{\alpha}_{u},\mathbb{P}_{\boldsymbol{\alpha}_{u}})du$* is well defined and non-decreasing;* 4. (iv)

The map $t\mapsto\mathbb{E}[\mathcal{S}_{t}^{\boldsymbol{\alpha}^{\star},i}]$ is constant for every $t\in\mathbb{R}^{\star}_{+}$;

Then $\boldsymbol{\alpha}^{\star}$ is a Nash equilibrium and $J^{i}(\boldsymbol{\alpha}^{\star})=\mathbb{E}[\mathcal{W}_{0}^{\boldsymbol{\alpha}^{\star},i}]$. Moreover, any other Nash-equilibrium $\tilde{\boldsymbol{\alpha}}$ such that $\mathbb{E}[\mathcal{W}_{0}^{\tilde{\boldsymbol{\alpha}},i}]=\mathbb{E}[\mathcal{W}_{0}^{{\boldsymbol{\alpha}^{\star}},i}]$ and $J^{i}(\tilde{\boldsymbol{\alpha}})=J^{i}({\boldsymbol{\alpha}^{\star}})$ for any $i\in\llbracket 1,n\rrbracket$ satisfies the condition (iv).

Proof.

The proof is exactly the same as in Lemma 2.1. ∎

Let us now describe the steps to follow in order to apply Lemma 3.1. Since they are similar to the ones in the finite-horizon case, we only report the main changes.

Steps 1-3

For each player $i\in\llbracket 1,n\rrbracket$ we still search for a random field $\{w_{t}^{i}(x,\overline{x}),t\in[0,T],x,\overline{x}\in\mathbb{R}^{d}\}$ of the form $w_{t}^{i}(x,\overline{x})=K_{t}^{i}.(x-\overline{x})^{\otimes 2}+\Lambda_{t}^{i}.\overline{x}^{\otimes 2}+2Y_{t}^{i\intercal}x+R^{i}_{t}$ for which the optimality principle in Lemma 3.1 now leads to the system:

[TABLE]

Notice that there are no terminal conditions anymore since we are in the infinite horizon case. The coefficients $\Phi^{i0},\Psi^{i0},\Delta^{i0}_{t},\Gamma_{t}^{i0}$ are defined in (17). The fourth step is exactly the same as in the finite horizon case.

Step 5

We now search for a fixed point of the best response functions. Let us define $\boldsymbol{Y}=(Y^{1},...Y^{n})$ and propose an ansatz in a feedback form $Y$: $Y_{t}=\pi(X_{t}-\mathbb{E}\left[X_{t}\right])+\hat{\pi}\mathbb{E}\left[X_{t}\right]+\eta_{t}$ where $\pi,\hat{\pi}\in L^{\infty}(\mathbb{R}_{+},\mathbb{R}^{nd\times d})$ and $\eta\in L^{2}_{\mathbb{F}}(\Omega\times\mathbb{R}_{+},\mathbb{R}^{nd})$ satisfy

[TABLE]

where the coefficients are defined in (22).

Step 6

We finally tackle the well-posedness of (44) and (45).

1. (i)

We first consider the ODE for $K^{i}$. Since the map $(t,k)\mapsto\phi_{t}^{i0}(k)$ does not depend on time (all the coefficient being constant) we search for a constant non-negative matrix $K^{i}\in{\mathbb{S}}^{d}$ satisfying $\Phi^{i0}(K^{i})=0$, more precisely solution to:

[TABLE]

As in [2] we can show using a limit argument that there exists $K^{i}\in{\mathbb{S}}_{+}^{d}$ solution to (46). The argument for $\Lambda^{i}$ is the same as for $K^{i}$. 2. (ii)

Given $(K^{i},\Lambda^{i})$ the equation for $(Y^{i},Z^{i})$ is a linear mean-field BSDE on infinite horizon:

[TABLE]

where the coefficient are defined in (33). Notice that now $\mathcal{G}^{i},\mathcal{\hat{G}}^{i},\mathcal{J}^{i},\mathcal{\hat{J}}^{i}$ are all constant matrices. To the best of our knowledge, there are no general results ensuring the existence for such equation. We then add the following assumption:

(H4’) There exists a solution $(Y^{i},Z^{i})\in\times\mathcal{S}_{\mathbb{F}}^{2}(\Omega\times\mathbb{R}_{+},\mathbb{R}^{d})\times L_{\mathbb{F}}^{2}(\Omega\times\mathbb{R}_{+},\mathbb{R}^{d})$ 3. (iii)

Given $(K^{i},\Lambda^{i},Y^{i},Z^{i})$ the equation for $R^{i}$ is a linear ODE whose solution is:

[TABLE]

where $h^{i}$ is a deterministic function defined in (34). 4. (iv)

We now study the well posedness of the fixed point procedure. More precisely we need to ensure that the process $t\to(\pi,\hat{\pi},\eta_{t})$ defined as a solution of the system (45), recalled below, is well defined. Note that in the infinite horizon framework we search for constant $\pi$ and $\hat{\pi}$.

[TABLE]

Existence of $(\pi,\hat{\pi})$ in whole generality is a difficult problem. Let us first rewrite the system (48) as:

[TABLE]

Where $\mathcal{C}=(b_{x},\sigma_{x},B,\Sigma,\hat{b}_{x},\hat{\sigma}_{x},\hat{B},\hat{\Sigma},S_{x},S_{y},S_{z},\hat{S}_{x},\hat{S}_{y},\hat{S}_{z},P_{y},P_{z},P_{\alpha},\hat{P}_{y},\hat{P}_{z},\hat{P}_{\alpha})$. Note that $F$ is continuously differentiable on its domain of definition. Thus, if $\left[\frac{\partial F}{\partial\pi},\frac{\partial F}{\partial\hat{\pi}}\right](\pi,\hat{\pi},\mathcal{C})$ is invertible for, then, by the implicit function theorem, there exists an open set $U$ containing $\mathcal{C}$ and a continuously differentiable function $g:U\mapsto(\mathbb{R}^{nd\times d})^{2}$ such that for all admissible coefficients $\mathcal{C}\in U$: $F(g(\mathcal{C}),\mathcal{C})=0$ and the solutions $(\pi,\hat{\pi})=g(\mathcal{C})$ are unique. It means that if we find a solution to (48) while the condition $\left[\frac{\partial F}{\partial\pi},\frac{\partial F}{\partial\hat{\pi}}\right]$ is invertible, then for small perturbations on the coefficients we still have solutions for $(\pi,\hat{\pi})$.

Let us now give sufficient conditions to ensure the existence of $(\pi,\hat{\pi})$ in a simplified setting where the state $t\to X_{t}$ belongs to $\mathbb{R}$ and all the players are symmetric in the sense that all the coefficients associated with each player are equals ($b_{1}=...=b_{2},Q^{1}=...=Q^{n},etc.$). We suppose also that the volatility is not controlled i.e. $\sigma_{i}=0$ for all $i\in\llbracket 1,n\rrbracket$. In such a case $\pi,\hat{\pi}\in\mathbb{R}^{n\times 1}$, $\pi_{1}=...=\pi_{n}$, $\hat{\pi}_{n}=...=\hat{\pi}_{n}$ and the systems (48) of coupled equations now reduces to two coupled second order equations:

[TABLE]

If we note:

[TABLE]

then a sufficient condition for $\pi_{1}$ to exists is simply: $b^{2}-4ac\geq 0$. Since $a\leq 0$ and $c\geq 0$ we have two possibilities a priori for $\pi_{1}$. We choose the positive one to ensure that $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$. Then if we note:

[TABLE]

a sufficient condition for $\hat{\pi}_{1}$ to exist is $\hat{b}^{2}-4\hat{a}\hat{c}(\pi_{1})\geq 0$. To ensure that there is a positive solution we also need $\hat{c}(\pi_{1})\geq 0$. 5. (v)

Let us finally verify that $\boldsymbol{\alpha}^{\star}\in\boldsymbol{\mathcal{A}}$. Let us consider the candidate for the optimal control for each player:

[TABLE]

where the coefficients are defined in (26) and $X^{\star}$ is the state process optimally controlled. Since $R,\hat{R}\in L^{2}_{\mathbb{F}}(\Omega\times\mathbb{R}_{+},\mathbb{R}^{\sum_{i}d_{i}})$ and given that the coefficient are constant in the infinite horizon case, we need to verify that:

[TABLE]

As we will see below, we will have to choose $\rho$ large enough to ensure these conditions. From the above expressions we see that $X^{\star}$ satisfies:

[TABLE]

with:

[TABLE]

where we define

[TABLE]

By Itô’s formula we have:

[TABLE]

If we now set:

[TABLE]

then, by Grownall inequality we obtain:

[TABLE]

Therefore, in order to have $\int_{0}^{\infty}e^{-\rho u}|\mathbb{E}[X_{u}^{\star}]|^{2}\mathrm{d}u<\infty$, we shall impose that $\rho>2\hat{B}^{\star}$. Finally, by Itô’s formula we also have:

[TABLE]

If we now set:

[TABLE]

then, by Grownall inequality we obtain:

[TABLE]

This time in order to ensure the convergence $\int_{0}^{\infty}e^{-\rho u}\mathbb{E}\left[|X_{u}^{\star}-\overline{X_{u}}^{\star}|^{2}\right]\mathrm{d}u<\infty$, we will add the constraint $\rho>2B^{\star}+4|\Sigma^{\star}|^{2}$. To conclude, in order to ensure that $\boldsymbol{\alpha}^{\star}\in\boldsymbol{\mathcal{A}}$ we make the following assumption:

(H5’) $\rho>\max\left[2\hat{B}^{\star},2B^{\star}+4|\Sigma^{\star}|^{2}\right]$.

3.2 The case of common noise

Let $W$ and $W^{0}$ be two independent Brownian motions defined on the same probability space $(\Omega,\mathcal{F},\mathbb{F},\mathbb{P})$ where $\mathbb{F}=\{\mathcal{F}_{t}\}_{t\in[0,T]}$ is the filtration generated by the pair $(W,W^{0})$. Let $\mathbb{F}^{0}=\{\mathcal{F}_{t}^{0}\}_{t\in[0,T]}$ be the filtration generated by $W^{0}$. For any $X_{0}$ and $\boldsymbol{\alpha}\in\boldsymbol{\mathcal{A}}$ as in Section 1, the controlled process $X^{\boldsymbol{\alpha}}$ is defined by:

[TABLE]

where for each $t\in[0,T]$, $x,\overline{x}\in\mathbb{R}^{d}$, $a_{i},\overline{a}_{i}\in\mathbb{R}^{d_{i}}$:

[TABLE]

Since we will condition on $W^{0}$, we assume that the coefficients $b_{x},\tilde{b}_{x},b_{i},\tilde{b}_{i},\sigma_{x},\tilde{\sigma}_{x},\sigma_{i},\tilde{\sigma}_{i},\sigma_{x}^{0},\tilde{\sigma}_{x}^{0},\sigma_{i}^{0},\tilde{\sigma}_{i}^{0}$ are essentially bounded and $\mathbb{F}^{0}$-adapted processes, whereas $\beta,\gamma,\gamma^{0}$ are square integrable $\mathbb{F}^{0}$-adapted processes. The problem of each player $i$ is to minimize over $\alpha_{i}$ $\in$ ${\cal A}^{i}$, and given control $\alpha^{-i}$ of the other players, a cost functional of the form

[TABLE]

with $f^{i},g^{i}$ as in (4). We now suppose that $Q^{i},\tilde{Q}^{i},I^{i},\tilde{I}^{i},N^{i},\tilde{N}^{i}$ are essentially bounded and $\mathbb{F}^{0}$-adapted, $L_{x}^{i},L^{i}_{k}$ are square-integrable $\mathbf{F}$-adapted processes, $P^{i},\tilde{P}^{i}$ are essentially bounded $\mathcal{F}_{T}^{0}$-mesurable random variable and $r^{i}$ are square-integrable $\mathcal{F}_{T}$-mesurable random variable. Hypothesis c) and d) of (H2) still holds. As in step 1 we guess a random field of the type $\mathcal{W}^{\boldsymbol{\alpha},i}_{t}=w_{t}^{i}(X_{t}^{\boldsymbol{\alpha}},\mathbb{E}\left[X_{t}^{\boldsymbol{\alpha}}|W_{t}^{0}\right])$ where $w^{i}$ is of the form $w_{t}^{i}(x,\overline{x})=K_{t}^{i}.(x-\overline{x})^{\otimes 2}+\Lambda_{t}^{i}.\overline{x}^{\otimes 2}+2Y_{t}^{i\intercal}x+R^{i}_{t}$ with suitable coefficients $K^{i},\Lambda^{i},Y^{i},R^{i}$. Given that the quadratic coefficient in $f^{i},g^{i}$ are $\mathbb{F}^{0}$-adapted we guess that $K^{i},\Lambda^{i}$ are also $\mathbb{F}^{0}$-adapted. Since the linear coefficients in $f^{i},g^{i}$ and the affine coefficients in $b,\sigma,\sigma^{0}$ are $\mathbb{F}$-adapted, we guess that $Y^{i}$ is $\mathbb{F}$-adapted as well. Thus for each player we look for processes $(K^{i},\Lambda^{i},Y^{i},R^{i})$ valued in $\mathbb{S}^{d}_{+}\times\mathbb{S}^{d}_{+}\times\mathbb{R}^{d}\times\mathbb{R}$ and of the form:

[TABLE]

where $\dot{K}^{i},\dot{\Lambda}^{i},Z^{K^{i}},Z^{\Lambda^{i}}$ are $\mathbb{F}^{0}$-adapted processes valued in $\mathbb{S}^{d}$; $\dot{Y}^{i},Z^{Y^{i}},Z^{0,Y^{i}}$ are $\mathbb{F}$-adapted processes valued in $\mathbb{R}^{d}$ and $R^{i}$ are continuous functions valued in $\mathbb{R}$. In step 2 we now consider, for each player $i\in\llbracket 1,n\rrbracket$, a family of processes of the form:

[TABLE]

By Itô’s formula we then obtain for (12) and (13):

[TABLE]

with

[TABLE]

Note that we now denote by $\overline{U}$ the conditional expectation with respect to $W_{t}^{0}$, i.e. $\overline{U}=\mathbb{E}\left[U|W_{t}^{0}\right]$. Then, at step 3, we constraint the coefficients $(K^{i},\Lambda^{i},Y^{i},R^{i})$ to satisfy the following problem:

[TABLE]

where $\Phi^{i0},\Psi^{i0},\Delta^{i0},\Gamma^{i0}$ are defined in (18). Thus we obtain the best response functions of the players:

[TABLE]

We then proceed to step 5 and to the search of a fixed point in the space of controls. The only difference at that point is in the ansatz for $t\mapsto\boldmath{Y_{t}}$. Since we consider the case of common noise, we now search for an ansatz of the form $\boldsymbol{Y}_{t}=\pi_{t}(X_{t}-\overline{X}_{t})+\hat{\pi}_{t}\overline{X}_{t}+\eta_{t}$ where $(\pi,\hat{\pi},\eta)\in L^{\infty}([0,T],\mathbb{R}^{nd\times d})\times L^{\infty}([0,T],\mathbb{R}^{nd\times d})\times\mathcal{S}_{\mathbb{F}}^{2}(\Omega\times[0,T],\mathbb{R}^{nd})$ satisfy:

[TABLE]

The method to determine the coefficients is then similar. Existence and uniqueness of a solution $(K^{i},\Lambda^{i})$ to the backward stochastic Ricatti equation in (61) is discussed in [13], section 3.2. The existence of a solution $(Y^{i},Z^{i},Z^{0,Y^{i}})$ to the linear mean-field BSDE in (61) is obtained as in step 6 thanks to Thm. 2.1 in [12]. As in the previous section the existence of a soltion $(\pi,\hat{\pi})\in L^{\infty}_{\mathbb{F}^{0}}(\Omega\times\mathbb{R}_{+},\mathbb{R}^{nd\times d})$ (essentially bounded $\mathbb{F}^{0}$-adapted functions) of (63) in the general case is a conjecture and needs to be verified in each example. We are not aware of any work tackling the existence of solutions in such situation. Given $(\pi,\hat{\pi})$ the existence of $(\eta,\phi,\phi^{0})$ solution to (63) is ensured as in the previous section by Thm. 2.1 in [12].

3.3 The case of multiple Brownian motions

We quickly sketch an extension to the case where there are multiple Brownian motions driving the state equation. The assumptions on the coefficients are the same as in the previous part. Only the length of the calculus changes. Let us now consider the state dynamic:

[TABLE]

where $\Phi^{i0},\Psi^{i0},\Delta^{i0},\Gamma_{t}^{i0}$ are defined in (18).

[TABLE]

We require the coefficients in (64) to satisfy an adaptation of (H1) where $\gamma,\sigma_{x},\tilde{\sigma}_{x},(\sigma_{i},\tilde{\sigma}_{i})_{i\in\llbracket 1,n\rrbracket}$ are replaced by $\gamma^{\ell},\sigma_{x}^{\ell},\tilde{\sigma}_{x}^{\ell},(\sigma_{i}^{\ell},\tilde{\sigma}_{i}^{\ell})_{i\in\llbracket 1,n\rrbracket}$ for $\ell\in\{1,...,\kappa\}$.

To take into account the multiple Brownian motions in step 1, we now search for random fields of the form $w_{t}^{i}(x,\overline{x})=K_{t}^{i}.(x-\overline{x})^{\otimes 2}+\Lambda_{t}^{i}.\overline{x}^{\otimes 2}+2Y_{t}^{i\intercal}x+R_{t}^{i}$ with the processes $(K^{i},\Lambda^{i},Y^{i},(Z^{\ell,i})_{\ell\in\llbracket 1,\kappa\rrbracket},R^{i})$ in $(L^{\infty}([0,T],\mathbb{S}^{d}_{+}))^{2}\times,\mathcal{S}^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{d})\times(L^{2}_{\mathbb{F}}(\Omega\times[0,T],\mathbb{R}^{d}))^{\kappa}\times L^{\infty}([0,T],\mathbb{R})$ and solution to:

[TABLE]

where $(\dot{K}^{i},\dot{\Lambda}^{i},\dot{R}^{i})$ are deterministic processes valued in $\mathbb{S}^{d}_{+}\times\mathbb{S}^{d}_{+}\times\mathbb{R}$ and $(\dot{Y}^{i},Z^{i})$ are adapted processes valued in $\mathbb{R}^{d}$.

The method then follows the same steps with generalized coefficients and at step 2 we obtain generalized coefficient for (12) and (13):

[TABLE]

[TABLE]

From these extended formulas we can then constrain the coefficients as in step 3 and obtain (18) with now the generalized coefficients defined in (66) and (67). The step 4 is then straightforward and we obtain the best response functions:

[TABLE]

From step 4 we can then continue to step 5, i.e. the fixed point search. The only difference at that point is in the ansatz for$t\mapsto\boldmath{Y_{t}}$. Since we consider the case with multiple Brownian motions we now search for an ansatz of the form $\boldsymbol{Y}_{t}=\pi_{t}(X_{t}-\overline{X}_{t})+\hat{\pi}_{t}\overline{X}_{t}+\eta_{t}$ where $(\pi,\hat{\pi},\eta)\in L^{\infty}([0,T],\mathbb{R}^{nd\times d})\times L^{\infty}([0,T],\mathbb{R}^{nd\times d})\times\mathcal{S}_{\mathbb{F}}^{2}(\Omega\times[0,T],\mathbb{R}^{nd})$ satisfy:

[TABLE]

The method to determine the coefficients $\pi,\hat{\pi},\eta$ is then similar. The validity of the computations i.e. Step 6 can be done exactly as in the case of a single brownian motion.

4 Example

We now focus on a toy example to illustrate the previous results. Let us consider a two player game where the state dynamics is simply a Brownian motion that two players can control. The goal of each player is to get the state near its own target $t\mapsto T^{i}_{t}$, where $t\mapsto T^{i}_{t}$, $i$ $=$ $1,2$, is a stochastic process. In order to add mean-field terms we suppose that each player try also to minimize the variance of the state and the variance of their controls.

[TABLE]

where $(\lambda^{i},\delta^{i},\theta^{i},\xi^{i})\in\mathbb{R}^{4}_{+}$. In order to fit to the context described in the first section we rewrite the cost function as follows:

[TABLE]

Since the terms $\delta^{i}(T^{i})^{2}$ do not influence the optimal control of the players, we work with the slightly simplified cost function:

[TABLE]

Following the method explained in the previous section, we use Theorem 2.3 in order to find a Nash equilibrium. We obtain the feedback form of the open loop controls and the dynamics of the state:

[TABLE]

where $K^{i}\in\mathbb{R}_{+},\Lambda^{i}\in\mathbb{R}_{+},a\in\mathbb{R},\tilde{a}\in\mathbb{R},\pi\in\mathbb{R}^{2},\tilde{\pi}\in\mathbb{R}^{2},\eta\in L^{2}_{\mathbb{F}}((0,\infty),\mathbb{R}^{2}),\gamma\in L^{2}_{\mathbb{F}}((0,\infty),\mathbb{R}^{2})$ satisfy:

[TABLE]

From (72) and (73) we can study and simulate the influence of the different parameters of the cost-function of the first player. We notice that $(\lambda^{1},\theta^{1})$ only influence $X-\mathbb{E}\left[X\right]$ and the feedback form of $\alpha_{1}-\overline{\alpha}_{1}$ (zero-mean terms).

• •

If $\lambda^{1}\to\infty$ then $\pi_{1}\sim\lambda^{1}$ and $K^{1}\to\infty$ which implies that $X_{t}-\mathbb{E}\left[X\right]_{t}\to 0$ for all $t\geq 0$. This is expected since the term $\lambda^{1}$ penalizes the variance of the state $\mathrm{Var}{(X_{t})}$ in the cost function of the first player. See Figure 1.

• •

If $\delta^{1}\to\infty$ then $(\pi,\tilde{\pi})\to\infty$ and $K^{1}\sim\delta^{1}$ and $\Lambda^{1}\sim\delta^{1}$ which imply that $X_{t}\to T^{1}_{t}$ for all $t\geq 0$. This is also expected since the term $\delta^{1}$ penalizes the quadratic gap between the state $X$ and the target $T^{1}$. See Figure 2.

• •

If $\theta^{1}\to\infty$ then $P^{1}\to 0$, $P^{1}K^{1}\to 0$, $P^{1}\pi^{1}\to 0$ and $P^{1}\eta^{1}_{t}\to 0$ for every $t\geq 0$. We then have $\alpha_{t}-\overline{\alpha}_{t}\to 0$ for all $t\geq 0$ and all the terms relative to the first player in $X-\mathbb{E}\left[X\right]$ disappear. Given that $\theta^{1}$ penalizes the variance of the control of the first player, this convergence is also intuitive.

• •

If $\xi^{1}\to\infty$ then $(P^{1},\tilde{P}^{1})\to 0$ which imply that $(\alpha_{1,t},\overline{\alpha}_{1,t})\to 0$ for all $t\geq 0$ and all the terms relative to the first player in $X_{t}-\mathbb{E}\left[X_{t}\right]$ and $\mathbb{E}\left[X_{t}\right]$ disappear for all $t\geq 0$. This means that the first player becomes powerless.