Linear-Quadratic Mixed Stackelberg-Nash Stochastic Differential Game with Major-Minor Agents

TL;DR

This paper develops a framework for analyzing a large-population stochastic differential game involving major and minor agents, deriving equilibrium strategies through FBSDEs and Riccati equations.

Contribution

It introduces a novel combined Stackelberg-Nash equilibrium approach for mixed agent populations using mean-field game theory and provides explicit feedback strategies.

Findings

Derivation of equilibrium strategies via FBSDEs.

Explicit feedback form of strategies using Riccati equations.

Application to large-scale multi-agent systems.

Abstract

We consider a controlled linear-quadratic (LQ) large-population system with mixture of three types agents: major leader, minor leaders and minor followers. The Stackelberg-Nash-Cournot (SNC) approximate equilibrium is studied by a major-minor mean-field game (MFG) coupled with a leader-follower Stackelberg game. By variational method, the SNC approximate equilibrium strategy can be represented by some forward-backward-stochastic-differential-equations (FBSDEs) in the open-loop sense. And we pay great effort to give the feedback form of the open-loop strategy by some Riccati equations.

Equations

$${\mathcal{A}}_0:\{\begin{array}{rl}& \mathrm{d}{X}_0(t)=\{A_0{X}_0(t)+B_0{u}_0(t)+E_0^1{X}^{(N_l)}(t)+F_0^1{x}^{(N_f)}(t)\}\mathrm{d}t\\ & +\{C_0{X}_0(t)+D_0{u}_0(t)+E_0^2{X}^{(N_l)}(t)+F_0^2{x}^{(N_f)}(t)\}\mathrm{d}{W}_0(t),\\ & X_0(0)={\xi }_0,\end{array}$$

$${\mathcal{A}}_i^l:i=1,2,\dots ,N_l,\{\begin{array}{rl}& \mathrm{d}{X}_i(t)=\{A{X}_i(t)+B{u}_i(t)+E_1{X}^{(N_l)}(t)\}\mathrm{d}t\\ & +\{C{X}_i(t)+D{u}_i(t)+E_2{X}^{(N_l)}(t)\}\mathrm{d}{W}_i(t)\\ & X_i(0)={\xi }_i\end{array}$$

$${\mathcal{A}}_j^f:j=1,2,\dots ,N_f,\{\begin{array}{rl}& \mathrm{d}{x}_j(t)=\{\tilde{A}{x}_j(t)+\tilde{B}{v}_j(t)+F_1{x}^{(N_f)}(t)\}\mathrm{d}t\\ & +\{\tilde{C}{x}_j(t)+\tilde{D}{v}_j(t)+F_2{x}^{(N_f)}(t)\}\mathrm{d}{\tilde{W}}_j(t)\\ & x_j(0)={\zeta }_j,\end{array}$$

$${\mathcal{J}}_0(u_0(\cdot ),\text{u}(\cdot ),\text{v}(\cdot ))=$$

$$\displaystyle\qquad\qquad+\|u_{0}(t)\|_{R_{0}}^{2}\Big{)}\mathrm{d}t+\|X_{0}(T)\|_{H_{0}}^{2}\Big{\}};$$

$${\mathcal{J}}_i^l(u_0(\cdot ),u_i(\cdot ),{\text{u}}_{-i}(\cdot ))=$$

$$\displaystyle\qquad\qquad+\|u_{i}(t)\|_{R}^{2}\Big{)}\mathrm{d}t+\|X_{i}(T)\|_{H}^{2}\Big{\}},$$

$${\mathcal{J}}_j^f(u_0(\cdot ),\text{u}(\cdot ),v_j(\cdot ),{\text{v}}_{-j}(\cdot ))=$$

$$\displaystyle\qquad\qquad+\|v_{j}(t)\|_{\widetilde{R}}^{2}\Big{)}\mathrm{d}t+\|x_{j}(T)\|_{\widetilde{H}}^{2}\Big{\}}.$$

$$\displaystyle L^{p}(t,T;\mathbb{H})=\Big{\{}\psi:[t,T]\rightarrow\mathbb{H}\ \Big{|}\ \int_{t}^{T}|\psi(s)|^{p}\mathrm{d}s<\infty\Big{\}},\qquad 1\leq p<\infty,$$

$$\displaystyle L^{\infty}(t,T;\mathbb{H})=\Big{\{}\psi:[t,T]\rightarrow\mathbb{H}\ \Big{|}\ \mbox{esssup}_{s\in[t,T]}|\psi(s)|<\infty\Big{\}},$$

$$\displaystyle C([t,T];\mathbb{H})=\Big{\{}\psi:[t,T]\rightarrow\mathbb{H}\ \Big{|}\ \psi(\cdot)\ \mathrm{is}\ \mathrm{continuous}\Big{\}}.$$

$$\displaystyle L^{2}_{{\cal F}_{t}}(\Omega;\mathbb{H})=\Big{\{}\xi:\Omega\rightarrow\mathbb{H}\ \Big{|}\ \xi\ \mathrm{is}\ {\cal F}_{t}\mathrm{-measurable},\ \mathbb{E}[|\xi|^{2}]<\infty\Big{\}},$$

$$\displaystyle L^{2}_{{\cal F}}(t,T;\mathbb{H})=\Big{\{}\psi:[t,T]\times\Omega\rightarrow\mathbb{H}\ \Big{|}\ \psi(\cdot)\ \mathrm{is}\ {\cal F}_{t}\mathrm{-progressively}\ \mathrm{measurable},\ \mathbb{E}\Big{[}\int_{t}^{T}|\psi(s)|^{2}\mathrm{d}s\Big{]}<\infty\Big{\}},$$

$$\displaystyle L^{2}_{{\cal F}}(\Omega;C([t,T];\mathbb{H}))=\Big{\{}\psi:[t,T]\times\Omega\rightarrow\mathbb{H}\ \Big{|}\ \psi(\cdot)\ \mathrm{is}\ {\cal F}_{t}\mathrm{-adapted},\ \mathrm{continuous},\ \mathbb{E}\Big{[}\sup_{t\leq s\leq T}|\psi(s)|^{2}\Big{]}<\infty\Big{\}},$$

$$\displaystyle L^{2}_{{\cal F}}(\Omega;L^{1}(t,T;\mathbb{H}))=\Big{\{}\psi:[t,T]\times\Omega\rightarrow\mathbb{H}\ \Big{|}\ \psi(\cdot)\ \mathrm{is}\ {\cal F}_{t}\mathrm{-progressively}\ \mathrm{measurable},\ \mathbb{E}\Big{[}\int_{t}^{T}|\psi(s)|\mathrm{d}s\Big{]}^{2}<\infty\Big{\}}.$$

$${\mathcal{U}}_0^c[0,T]\triangleq$$

$${\mathcal{U}}_i^c[0,T]\triangleq$$

$${\mathcal{V}}_j^c[0,T]\triangleq$$

$${\mathcal{U}}_0^d[0,T]\triangleq$$

$${\mathcal{U}}_i^d[0,T]\triangleq$$

$${\mathcal{V}}_j^d[0,T]\triangleq$$

$$\{\begin{array}{rlrl}& {\mathcal{U}}^c[0,T]={\mathcal{U}}_1^c[0,T]\times \cdots \times {\mathcal{U}}_{N_l}^c[0,T],& & {\mathcal{V}}^c[0,T]={\mathcal{V}}_1^c[0,T]\times \cdots \times {\mathcal{V}}_{N_f}^c[0,T],\\ & {\mathcal{U}}^d[0,T]={\mathcal{U}}_1^d[0,T]\times \cdots \times {\mathcal{U}}_{N_l}^d[0,T],& & {\mathcal{V}}^d[0,T]={\mathcal{V}}_1^d[0,T]\times \cdots \times {\mathcal{V}}_{N_f}^d[0,T],\end{array}$$

$$\{\begin{array}{rl}& {\mathcal{U}}_{-i}^c[0,T]={\mathcal{U}}_1^c[0,T]\times \cdots \times {\mathcal{U}}_{i-1}^c[0,T]\times {\mathcal{U}}_{i+1}^c[0,T]\times \cdots \times {\mathcal{U}}_{N_l}^c[0,T],\\ & {\mathcal{V}}_{-j}^c[0,T]={\mathcal{V}}_1^c[0,T]\times \cdots \times {\mathcal{V}}_{j-1}^c[0,T]\times {\mathcal{V}}_{j+1}^c[0,T]\times \cdots \times {\mathcal{V}}_{N_f}^c[0,T],\\ & {\mathcal{U}}_{-i}^d[0,T]={\mathcal{U}}_1^d[0,T]\times \cdots \times {\mathcal{U}}_{i-1}^d[0,T]\times {\mathcal{U}}_{i+1}^d[0,T]\times \cdots \times {\mathcal{U}}_{N_l}^d[0,T],\\ & {\mathcal{V}}_{-j}^d[0,T]={\mathcal{V}}_1^d[0,T]\times \cdots \times {\mathcal{V}}_{j-1}^d[0,T]\times {\mathcal{V}}_{j+1}^d[0,T]\times \cdots \times {\mathcal{V}}_{N_f}^d[0,T].\end{array}$$

$$\{\begin{array}{rl}& A_0,A,\tilde{A},C_0,C,\tilde{C},E_0^1,E_0^2,E_1,E_2,F_0^1,F_0^2,F_1,F_2\in {\mathbb{R}}^{n\times n};\\ & B_0,D_0\in {\mathbb{R}}^{n\times m_1};B,D\in {\mathbb{R}}^{n\times m_2};\tilde{B},\tilde{D}\in {\mathbb{R}}^{n\times m_3}.\\ & Q_0,Q,\tilde{Q},H_0,H,\tilde{H}\in {\mathcal{S}}^n;\\ & R_0\in {\mathcal{S}}^{m_1};R\in {\mathcal{S}}^{m_2};\tilde{R}\in {\mathcal{S}}^{m_3}.\end{array}$$

$${\mathcal{J}}_j^f(u_0(\cdot ),\text{u}(\cdot ),{\overline{v}}_j[u_0(\cdot ),\text{u}(\cdot ),{\xi }_0,{\xi }_i,{\zeta }_j](\cdot ),{\text{v}}_{-j}(\cdot ))$$

$$=\underset{v_j(\cdot )\in {\mathcal{V}}_j^c[0,T]}{\min }{\mathcal{J}}_j^f(u_0(\cdot ),\text{u}(\cdot ),v_j(\cdot ),{\text{v}}_{-j}(\cdot )),\forall u_0(\cdot )\in {\mathcal{U}}_0^c[0,T],\text{u}(\cdot )\in {\mathcal{U}}^c[0,T],$$

$${\mathcal{J}}_0({\overline{u}}_0[\text{u}(\cdot ),{\xi }_0,{\xi }_i,{\zeta }_j](\cdot ),\text{u}(\cdot ),\overline{\text{v}}[{\overline{u}}_0[\text{u}(\cdot ),{\xi }_0,{\xi }_i,{\zeta }_j](\cdot ),\text{u}(\cdot ),{\xi }_0,{\xi }_i,{\zeta }_j](\cdot ))$$

$$=\underset{u_0(\cdot )\in {\mathcal{U}}_0^c[0,T]}{\min }{\mathcal{J}}_0(u_0(\cdot ),\text{u}(\cdot ),\overline{\text{v}}[u_0(\cdot ),\text{u}(\cdot ),{\xi }_0,{\xi }_i,{\zeta }_j](\cdot )),\forall \text{u}(\cdot )\in {\mathcal{U}}^c[0,T],$$

$${\mathcal{J}}_i^l({\overline{u}}_0[{\overline{u}}_i(\cdot ),{\text{u}}_{-i}(\cdot ),{\xi }_0,{\xi }_i,{\zeta }_j](\cdot ),{\overline{u}}_i(\cdot ),{\text{u}}_{-i}(\cdot ))$$

$$=\underset{u_i(\cdot )\in {\mathcal{U}}_i^c[0,T]}{\min }{\mathcal{J}}_i^l({\overline{u}}_0[\text{u}(\cdot ),{\xi }_0,{\xi }_i,{\zeta }_j](\cdot ),u_i(\cdot ),{\text{u}}_{-i}(\cdot )),$$

$${\mathcal{J}}_j^f({\xi }_0,{\zeta }_j,{\overline{m}}_X(\cdot );u_0(\cdot ),\text{u}(\cdot ),v_j(\cdot ))=$$

$$\displaystyle\qquad\qquad+\|v_{j}(t)\|_{\widetilde{R}}^{2}\Big{)}\mathrm{d}t+\|x_{j}(T)\|_{\widetilde{H}}^{2}\Big{\}},$$

$$\{\begin{array}{rl}& \mathrm{d}{\overline{x}}_j(t)=\{\tilde{A}{\overline{x}}_j(t)+\tilde{B}{v}_j(t)+F_1{\overline{m}}_x(t)\}\mathrm{d}t+\{\tilde{C}{\overline{x}}_j(t)+\tilde{D}{v}_j(t)+F_2{\overline{m}}_x(t)\}\mathrm{d}{\tilde{W}}_j(t)\\ & {\overline{x}}_j(0)={\zeta }_j,\end{array}$$

$$J_j^f({\xi }_0,{\zeta }_j,{\overline{m}}_X(\cdot ),{\overline{m}}_x(\cdot );u_0(\cdot ),v_j(\cdot ))=$$

$$\displaystyle\qquad\qquad+\|v_{j}(t)\|_{\widetilde{R}}^{2}\Big{)}\mathrm{d}t+\|x_{j}(T)\|_{\widetilde{H}}^{2}\Big{\}},$$

$$J_j^f({\xi }_0,{\zeta }_j,{\overline{m}}_X(\cdot ),{\overline{m}}_x(\cdot );u_0(\cdot ),{\overline{v}}_j[u_0(\cdot ),{\overline{m}}_X(\cdot ),{\overline{m}}_x(\cdot ),{\xi }_0,{\zeta }_j])=\underset{v_j(\cdot )\in {\mathcal{V}}_j^d[0,T]}{\inf }{J}_j^f({\xi }_0,{\zeta }_j,{\overline{m}}_X(\cdot ),{\overline{m}}_x(\cdot );u_0(\cdot ),v_j(\cdot )).$$

$$\mathbb{E}\Big{[}\overline{x}_{j}\big{(}\overline{v}_{j}[u_{0}(\cdot),\overline{m}_{X}(\cdot),\overline{m}_{x}(\cdot),\xi_{0},\zeta_{j}]\big{)}\Big{|}{\cal F}_{t}^{0}\Big{]}=\overline{m}_{x}.$$

$$\{\begin{array}{rl}& \mathrm{d}{\overline{X}}_0(t)=\{A_0{\overline{X}}_0(t)+B_0{u}_0(t)+E_0^1{\overline{m}}_X(t)+F_0^1{\overline{m}}_x(t)\}\mathrm{d}t\\ & +\{C_0{\overline{X}}_0(t)+D_0{u}_0(t)+E_0^2{\overline{m}}_X(t)+F_0^2{\overline{m}}_x(t)\}\mathrm{d}{W}_0(t),\\ & {\overline{X}}_0(0)={\xi }_0,\\ & \end{array}$$

Full text

**Key words: ** Stackelberg-Nash-Cournot approximate equilibrium, Mean-field game, FBSDE, Leader-follower game, Major-minor agent, Open-loop strategy, Closed-loop strategy.

1 Introduction

On a given finite time horizon $[0,T],$ let $(\Omega,{\cal F},\mathbb{P})$ be a complete probability space on which a $(1+N_{l}+N_{f})$-dimensional standard Brownian motion $\{W_{0}(t),W_{i}(t)$, $\widetilde{W}_{j}(t)\}_{0\leq t\leq T}$ is defined. In this paper, we consider a large-population system involving $(1+N_{l}+N_{f})$ individual agents (where $N_{l}$ and $N_{f}$ are very large) which are mixed with three types: the major-leader, denoted by ${\cal A}_{0}$, minor-leaders ${\cal A}_{i}^{l},1\leq i\leq N_{l}$ and the followers ${\cal A}_{j}^{f},1\leq j\leq N_{f}$. The dynamics of ${\cal A}_{0},\{{\cal A}_{i}^{l}\}_{i=1}^{N_{l}},\{{\cal A}_{j}^{f}\}_{j=1}^{N_{f}}$ are given respectively by the following controlled linear stochastic differential equations:

[TABLE]

[TABLE]

and

[TABLE]

where $X^{(N_{l})}(t)=\frac{1}{N_{l}}\sum_{i=1}^{N_{l}}X_{i}(t)$ and $x^{(N_{f})}(t)=\frac{1}{N_{f}}\sum_{j=1}^{N_{f}}x_{j}(t)$ are called state-average or mean field term. $A_{0}$, $A$, $\widetilde{A}$, $B_{0}$, $B$, $\widetilde{B}$, $C_{0}$, $C$, $\widetilde{C}$, $D_{0}$, $D$, $\widetilde{D}$, $E_{0}^{1}$, $E_{0}^{2}$, $E_{1}$, $E_{2}$, $F_{0}^{1}$, $F_{0}^{2}$, $F_{1}$, $F_{2}$ are deterministic constant matrices with proper dimensions. In the above, $X_{0}(\cdot)$, $X_{i}(\cdot)$, $x_{j}(\cdot)$ are called the state process taking values in $\mathbb{R}^{n}$ with initial values $\xi_{0}$, $\xi_{i}$, $\zeta_{j}$ which are random variables. $u_{0}(\cdot)$, $u_{i}(\cdot)$, $v_{j}(\cdot)$ are called admissible controls taken by $(1+N_{l}+N_{f})$ players in the game and taking values in $\mathbb{R}^{m_{1}}$, $\mathbb{R}^{m_{2}}$, $\mathbb{R}^{m_{3}}$, respectively. Under some mild conditions on the coefficients, for any initial values $\xi_{0}$, $\xi_{i}$, $\zeta_{j}$, (LABEL:C6e1), (LABEL:C6e2) and (LABEL:C6e3) admits a unique strong solution. The performance can be measured by the following cost functionals: for ${\cal A}_{0},$

[TABLE]

for ${\cal A}_{i}^{l},\ 1\leq i\leq N_{l}$;

[TABLE]

and for ${\cal A}_{j}^{f},\ 1\leq j\leq N_{f}$,

[TABLE]

where for given vector $z$, $\|z\|^{2}_{M}=\langle Mz,z\rangle$ for $M$ is any matrix or matrix-valued function of suitable dimensions, and where $Q_{0},Q,\widetilde{Q},R_{0},R,\widetilde{R},H_{0},H,\widetilde{H}$ are deterministic symmetric matrix of suitable dimensions. As we can see, all agents are coupled not only in their state process but also in their cost functionals with convex combinations of state-average.

Roughly speaking, agent ${\cal A}_{j}^{f}$ will give his/her best respond according to the strategies from major leader ${\cal A}_{0}$ and minor leaders ${\cal A}_{i}^{l}$ to minimize his/her own cost functional ${\cal J}_{j}^{f}(u_{0}(\cdot),\textbf{u}(\cdot),v_{j}(\cdot),\textbf{v}_{-j}(\cdot))$. And agent ${\cal A}_{0}$ will also give his/her best respond according to the strategies minor leaders ${\cal A}_{i}^{l}$ and the best respond of minor followers to minimize his/her own cost functional ${\cal J}_{0}(u_{0}(\cdot),\textbf{u}(\cdot),\textbf{v}(\cdot))$. Knowing the best respond of major leader and minor followers, agent ${\cal A}_{i}^{l}$ wants to minimize his/her own cost functional ${\cal J}_{i}^{l}(u_{0}(\cdot),u_{i}(\cdot),\textbf{u}_{-i}(\cdot))$ by choosing an optimal control $u_{i}(\cdot)$. However, due to the state-average coupling, our problem is essentially a high-dimensional Stackelberg-Nash differential game. Moreover, $\mathcal{A}_{0}$ is the the dominate or major leader because it effects the cost functionals of all minor leaders.

We call the above problem formulated as Mixed Stakelberg-Nash Major-minor (SN-MM) differential game. The following comments on our formulation further verify such terminology.

(Single leader-follower game) In case $N_{l}=0,N_{f}=1,$ thus there has no minor leaders and only single followers, with one major leader, then our problem reduces to the classical single-leader and single-follower game. The Stackelberg game has been proposed in 1934 by H. von Stackelberg [30], when he defined a concept of a hierarchical solution for markets where some firms have power of dominating over others. This solution concept is now known as the Stackelberg equilibrium. Early study for stochastic Stackelberg differential games (SSDG) can be seen in Basar (1979) [2]. A pioneer work was done by Yong (2002) [32], where a LQ leader-follower stochastic differential game (SDG) was introduced and studied. The coefficients of the system and the cost functionals are random, the controls enter the diffusion term of the state equation, and the weight matrices for the controls in the cost functionals are not necessarily positive definite. To give a state feedback representation of the OL Stackelberg equilibrium, the related Riccati equations are derived and sufficient conditions for the existence of their solution with deterministic coefficients are discussed. Here after, Bensoussan, Chen, and Sethi (2015) [4] obtained the global maximum principles for both open-loop (OL) and closed-loop (CL) SSDG whereas the diffusion term does not contain the controls. The solvability of related Riccati equations is discussed, in order to obtain the state feedback Stackelberg equilibrium.

(Multiple leaders-followers game) In case $N_{l},N_{f}$ are of medium or small size, then our problem is reduced to the Stackelberg game with multiple leaders and multiple followers. It is a natural extension of the single leader-follower game and the relevant works include [5, 6, 28], etc.

(Mean-field-game with symmetric agents) In case $N_{l}=0,$ and no $\mathcal{A}_{0}$ involved, then our problem becomes the standard dynamic game with a very large number of minor (symmetric) agents in which each single agent interacts with the mass-effect of other agents only through coupling in states/dynamics. For large population stochastic dynamics, one effective method is to search its decentralized strategies by the mean-field-game (MFG) theory. We recall that there are much work to study mean field game (MFG). Since the recent independent works by Huang, Caines, and Malhamé [19, 20] and Lasry and Lions [22, 23, 24], mean field game (MFG) theory and its applications have enjoyed rapid growth. MFG provides a simpler alternative framework for tackling the interactive game for a large number of homogeneous agents. By allowing agents to interact through a common medium, known as the mean field term, formulation of the dynamic game under the MFG framework consists of only a few equations. Further developments on the theory of MFG can be found in the works of Andersson and Djehiche [1], Bardi [3], Bensoussan, Frehse, and Yam [8], Buckdahn et al. [7], Cardaliaguet [10], Carmona and Delarue [11], Garnier, Papanicolaou, and Yang [15], Guéant, Lasry, and Lions [16], Meyer-Brandis, Øksendal, and Zhou [25], and the references therein.

(Major-minor game) In case $N_{f}=0,$ then there has no followers and only major and minor leaders involved, and our problem becomes the major-minor (MM) mean-field game (MFG). The MM-MFG is introduced in [21], and has been well investigated by [27] common major-minor mean field LQG game (refer to [27]). Our model generalizes [5, 6, 27, 28] because it includes not only leader-follower structure but also major-minor structure.

(Convex combination) Refer to Nourian, Caines, Malhamé and Huang (2012) [28], here we consider a kind of general case of cost functional with likelihood ratio (i.e. convex combination). On other words, for an example, the cost functional of the major leaders is based on a trade-off between keeping cohesion of the flock of minor leaders and keeping cohesion of the flock of the followers (see (4)). By the way, we may be interested in special case like $\widetilde{\lambda}_{3}\neq 0$ which means the cost functional of followers are directly influenced by the major-leader or $\widetilde{\lambda}_{3}=0$ which means the cost functional of followers are not directly influenced by the major-leader. We will discuss difference between the special case and the general case in following sections.

Remark 1.1**.**

Application of our problem formulation may be found in power markets involving large size of consumers and large utilities together with the following producer; inventory management without stocking capacities. (refer to [13]) The state processes are characterized by three kinds of group. One we called major leader agent can be regarded as the government or supervisory in the economic issues. And the ones we called minor leader agents can be regarded as the corresponding companies or firms. The rest ones we called minor follower agents can be regarded as the related suppliers of raw material or manufacturers of primary commodity, etc. We can see that the state processes of three types of group have no influence on each other but the cost functionals do have direct influence on each other.

Our present work considers the combination problems of leader-follower and major-minor systems, where the large scale population is also under consideration. In the entire system, the major and a part of minor agents are together regarded as the leaders, which are called major-leader and minor-leaders respectively and the rest are called minor followers (followers). Obviously, the more complex structure will bring some technical problem. Besides, there are lots of interesting questions remain to be solved. For an example, we can consider the state processes include the mean-field term which may coincide the real world much better or focus on the more realistic cost functional, etc.

Let us now explain the argument structure of our problem. In principle, the above problem can be studied as a MM-MFG coupled with a leader-follower game. Accordingly, original problem can be analyzed through the following structures:

Step 1: Fix the mass effect limit of minor leaders $\overline{m}_{X}$ and major leader $(x_{0},u_{0}).$ With frozen $(x_{0},u_{0},\overline{m}_{X}),$ introduce and solve the auxiliary problem to get the best decentralized response function of minor followers, denoted by $\overline{m}_{x}=\overline{m}_{x}(x_{0},u_{0},\overline{m}_{X}).$

Step 2: Given the response functional $\overline{m}_{x},$ and frozen $\overline{m}_{X},$ solve the decentralized SOC problem of $\mathcal{A}_{0},$ and denote the optimal solution pair as $(\overline{X}_{0},\overline{u}_{0})$.

Step 3: Given $\overline{m}_{x},$ solve the optimal control for the minor leaders. Influenced by the optimal control of major leader $\overline{u}_{0}(\cdot)$ (supposing it exists, which depends on the choices $\textbf{u}(\cdot)$ of minor leaders and the initial state $\xi_{0}$, $\xi_{i}$, $\zeta_{j}$, in general), agents ${\cal A}_{i}^{l}$, $1\leq i\leq N_{l}$, (the minor leaders) would like to choose some $\overline{u}_{i}(\cdot)$ to minimize ${\cal J}_{i}^{l}(\overline{u}_{0}(\cdot),u_{i}(\cdot),\textbf{u}_{-i}(\cdot))$.

Step 4 CC condition to specify $\overline{m}_{X}$ and all decentralized strategies can be designed. Approximate Stackelberg-Counot-Nash equilibrium can be verified.

The main contribution of this paper can be summarized as follows:

• •

The decentralized strategy profile is investigated in both (semi-)closed-loop and open-loop sense.

• •

Existence and uniqueness of the CC condition system is investigated in the global solvability case.

• •

the CC condition system is represented via a full-coupled mean-field type FBSDE in open-loop case, and FBSDE and non-standard Riccati equation in closed-loop sense.

• •

The approximate Nash equilibrium Stakleberg game is verified under more general condition (more than standard assumption with positive-definitiveness on coefficient matrix).

The rest of this paper is organized as follows. In section 2, we give the formal problem formulation and some preliminaries. In section 3, we discuss the open-loop strategy of Stackelberg mixed major-minor games. In section 4 We get the consistency condition system equations based on the open-loop strategy, which is a fully coupled FBSDE. Besides, we get the criteria to judge the well-posedness of such a FBSDE. At last, we verify the OL strategy we got is $\varepsilon$-Nash equilibrium OL strategy of the original problem.

2 Preliminary and formulation

The following notations will be used throughout this paper. Let $\mathbb{R}^{n}$ denotes the $n-$dimensional Euclidean space, $\mathbb{R}^{n\times m}$ be the set of all $(n\times m)$ matrices, and let ${\cal S}^{n}$ be the set of all $(n\times n)$ symmetric matrices. We denote the transpose by subscript ⊤, the inner product by $\langle\cdot,\cdot\rangle$ and the norm by $|\cdot|$. For $t\in[0,T]$ and Euclidean space $\mathbb{H}$, we introduce the following function spaces:

[TABLE]

and the spaces of process or random variables on given filtrated probability space:

[TABLE]

We set the following information structures, which are important to introduce our admissible strategies: $\{{\cal F}_{t}\}_{0\leq t\leq T}$ is the natural filtration generated by all BM components $\{W_{0}(\cdot),W_{i}(\cdot),\widetilde{W}_{j}(\cdot)\}$ augmented by all the $\mathbb{P}$-null sets in ${\cal F}$, $1\leq i\leq N_{l},\ 1\leq j\leq N_{f}$, it can be viewed as the full information of all states and noises; $\{{\cal F}_{t}^{0}\}_{0\leq t\leq T}$ is the natural filtration generated by $\{W_{0}(\cdot),X_{0}(\cdot)\}$ augmented by all the $\mathbb{P}$-null sets in ${\cal F}$. It is the space on which the limiting state-average should be adapted; $\{{\cal F}_{t}^{i}\}_{0\leq t\leq T}$ is the natural filtration generated by $\{W_{i}(\cdot),X_{i}(\cdot)\}$ augmented by all the $\mathbb{P}$-null sets in ${\cal F}$, $1\leq i\leq N_{l}$; $\{{\cal G}_{t}^{j}\}_{0\leq t\leq T}$ is the natural filtration generated by $\{\widetilde{W}_{j}(\cdot),Y_{j}(\cdot)\}$ augmented by all the $\mathbb{P}$-null sets in ${\cal F}$, $1\leq j\leq N_{f}$.

Given information structures, we can set the following Hilbert spaces for centralized and decentralized strategies for individual agents in open-loop sense:

[TABLE]

and decentralized open-loop strategies:

[TABLE]

Let $(u_{0},\textbf{u},\textbf{v})=(u_{0},u_{1},\ldots,u_{N_{l}},v_{1},\ldots,v_{N_{f}})$ denote the strategy set of all $(1+N_{l}+N_{f})$ agents; $\textbf{u}=(u_{1},\ldots,u_{N_{l}})$ the set of control strategies of all $N_{l}$ major-leader agents; $\textbf{v}=(v_{1},\ldots,v_{N_{f}})$ the set of strategy profile of all $N_{f}$ follower agents; $\textbf{u}_{-i}=(u_{1},\ldots,u_{i-1},$ $u_{i+1},\ldots,$ $u_{N_{l}})$ the control strategy set of major-leader agents except ${\cal A}_{i}^{l}$; $\textbf{v}_{-j}=(v_{1},\ldots,v_{j-1},$ $v_{j+1},\ldots,$ $v_{N_{f}})$ the control strategy set of follower agents except the $j^{th}$ follower agent ${\cal A}_{j}^{f}$.

Sometimes, when defining the Stakelberg-Nash-Counor strategy, it is helpful to set the following product space for strategy set. We denote

[TABLE]

[TABLE]

Then any $(u_{0},\textbf{u},\textbf{v})\in{\cal U}_{0}^{c}[0,T]\times{\cal U}^{c}[0,T]\times{\cal V}^{c}[0,T]$ is called an admissible centralized strategy, and any $(u_{0},\textbf{u},\textbf{v})\in{\cal U}_{0}^{d}[0,T]\times{\cal U}^{d}[0,T]\times{\cal V}^{d}[0,T]$ is called an admissible decentralized strategy.

Let us introduce the following hypothesis on coefficients of state dynamics and cost functionals:

(H1)

The coefficients of the state equations and cost functionals satisfy the following:

[TABLE]

(H2)

The initial states $\xi_{0},\xi_{i},\zeta_{j}\in L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})$ are independent; $\mathbb{E}[\xi_{i}]=\mathbb{E}[\zeta_{j}]=0$, for each $i=1,\ldots,N_{l}$, $j=1,2,\ldots,N_{f}$; and there exists $c_{0}<\infty$ independent of $N_{l}$ and $N_{f}$ such that $\sup_{i\geq 0}\mathbb{E}[|\xi_{i}|^{2}]\leq c_{0}$ and $\sup_{j\geq 1}\mathbb{E}[|\zeta_{j}|^{2}]\leq c_{0}$.

We point out that no positive-definiteness/non-negativeness conditions on the weighting matrix/matrix-valued functions imposed in (H1). Moreover, the coefficients of the convex combination $0\leq\lambda_{0}$, $\lambda$, $\widetilde{\lambda}_{1}$, $\widetilde{\lambda}_{2}$, $\widetilde{\lambda}_{3}\leq 1$. Under (H1), for any $(x_{0},u_{0}(\cdot))\in\mathbb{R}^{n}\times{\cal U}_{0}^{c}[0,T]$ (resp., $\mathbb{R}^{n}\times{\cal U}_{0}^{d}[0,T]$), $(x,u_{i}(\cdot))\in\mathbb{R}^{n}\times{\cal U}_{i}^{c}[0,T]$ (resp., $\mathbb{R}^{n}\times{\cal U}_{i}^{d}[0,T]$), $(y,v_{j}(\cdot))\in\mathbb{R}^{n}\times{\cal V}_{j}^{c}[0,T]$ (resp., $\mathbb{R}^{n}\times{\cal V}_{j}^{d}[0,T]$), (LABEL:C6e1), (LABEL:C6e2), (LABEL:C6e3) admits a unique (strong) solution. And the cost functionals (4), (5), (6) are also well-defined.

For simplicity, in (H2) it is assumed that all minor leaders and followers have zero initial mean. It is possible to generalize our analysis to deal with different initial means as long as $\{\mathbb{E}[\xi_{i}],i\geq 1\}$ and $\{\mathbb{E}[\zeta_{j}],j\geq 1\}$ has a limiting empirical distribution. Now, we can introduce the Stakelberg-Counot-Nash equilibrium as follows.

Definition 2.1**.**

A $(1+N_{l}+N_{f})$-tuple $(\overline{u}_{0}[\cdot],\overline{\textbf{u}}(\cdot),\overline{\textbf{v}}[\cdot])$, is called an open-loop Stakelberg-Counot-Nash equilibrium for the initial states $\xi_{0},\xi_{i},\zeta_{j}\in L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})$ if:

[TABLE]

where $\overline{v}_{j}:{\cal U}_{0}^{c}[0,T]\times{\cal U}^{c}[0,T]\times L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})\times L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})\times L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})\rightarrow{\cal V}_{j}^{c}[0,T]$, and $\overline{u}_{0}:{\cal U}^{c}[0,T]\times L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})\times L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})\times L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})\rightarrow{\cal U}_{0}^{c}[0,T]$.

If there is no confusion, we use the same notation to denote the optimal respond and the optimal strategy of major leader and minor followers. The above definition of OL strategy is defined in centralized sense. In particular, game theory has been formulated to capture such individual interest seeking behavior of the agents in many social, economic and man-made systems.

For fixed $N=N_{l}+N_{f}$, if each agents can access the full information (states) of other agents, we may view the problem as a standard dynamic LQG leader-follower games and use the full information DPP to derive the Stakelberg-Counot-Nash equilibrium. We now introduce the following definition. However, scale dynamic model, this approach results in an analytic complexity which is in general prohibitively high, and correspondingly leads to few substantive dynamic optimization results. The optimization of large-scale linear control systems wherein i) many agents are coupled with each other via their individual dynamics, and ii) the costs are in an individual to the mass form was presented in where the theory of mean field (MF) control (previously termed Nash Certainty Equivalence) was introduced. It is to be noted that the dynamic large-scale cost coupled optimization structure of is motivated by a variety of scenarios, for instance, those analyzed in MFG analysis.

2.1 Mixed Stakelberg-Counot-Nash equilibrium analysis

To deal with mixed leader-follower MM dynamic game using MFG theory, one should start with followers. And to deal with a major-minor MFG, one should start with major players. Although the relationships get complicated under our situation, we can still deal with it step by step. That is, firstly, we can solve the optimization problems of followers. The left is a classic major-minor problem and solved in the way of [21]. The interesting things occur when the major-leader imposes some direct impacts to the followers (i.e., $\widetilde{\lambda}_{3}\neq 0$), which will lead to that the state process of major leader will be relied on a kind of forward-backward stochastic differential equation (FBSDE). Generally speaking, it is hard to get the centralized strategy of such mixed Stackelberg MM-MFG. So, let us briefly look at the procedure of finding a decentralized open-loop $\varepsilon$-Nash equilibrium strategy of the original problem. And the procedure of finding a decentralized closed-loop $\varepsilon$-Nash equilibrium strategy is very similar which will be formulated in next subsection.

Step 1: MFG analysis of followers: Let us introduce the auxiliary limiting LQG differential game problems. Firstly, by the Stackelberg game, for given strategy of major leader and minor leaders, followers have to minimize the following cost functionals:

[TABLE]

where $\overline{m}_{X}(\cdot)=\lim_{N_{l}\rightarrow+\infty}X^{(N_{l})}(\cdot)$. Furthermore, as $N_{f}\rightarrow+\infty,$ we suppose $x^{(N_{f})}(\cdot)$ can be approximated by $\mathcal{F}^{0}_{t}$-measurable function $\overline{m}_{x}(\cdot)$. Then the state process of the follower becomes

[TABLE]

with the following auxiliary cost functionals

[TABLE]

for ${\cal A}_{j}^{f},\ 1\leq j\leq N_{f}$. To distinguish from the original problem, we use the new state variables $\overline{x}_{j}$ and we will denote $\overline{X}_{0}$ and $\overline{X}_{i}$ the new state variables later. But we still use the same set of variables $u_{0},u_{i},v_{j},W_{0},W_{i},\widetilde{W}_{j}$ in this auxiliary limiting problem, and such a reuse of notation should cause no confusion. Then, introduce the following auxiliary Nash game for followers as follows.

Problem (OL1). For given $\xi_{0},\zeta_{j}\in L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})$, ${\cal F}^{0}_{t}$-measurable functions $\overline{m}_{X}(\cdot),\overline{m}_{x}(\cdot)$, and the control $u_{0}(\cdot)$ of major leader ${\cal A}_{0}$, find an open-loop strategy $\overline{v}_{j}(\cdot)=\overline{v}_{j}[u_{0}(\cdot),\overline{m}_{X}(\cdot),\overline{m}_{x}(\cdot),\xi_{0},\zeta_{j}]\in{\cal V}_{j}^{d}[0,T]$, $1\leq j\leq N_{f}$. On other words, Find the Nash equilibrium response functional $\overline{v}_{j}[\cdot]:{\cal U}_{0}^{d}[0,T]\times L^{2}_{{\cal F}^{0}}(0,T;\mathbb{R}^{n})\times L^{2}_{{\cal F}^{0}}(0,T;\mathbb{R}^{n})\times L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})\times L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})\rightarrow{\cal V}_{j}^{d}[0,T]$ of the following Nash differential games among followers:

[TABLE]

The analysis of Problem (OL1) can be further decomposed into substeps using MFG theory.

Step 1.1 (SOC-F): Fixed $\overline{m}_{x}$, and consider the Nash equilibrium response functional of the above Problem (OL1) for representative minor-follower agent denoted by $\overline{v}_{j}[\cdot]$. For given $\xi_{0},\zeta_{j}\in L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})$, ${\cal F}^{0}_{t}$-measurable functions $\overline{m}_{X}(\cdot),\overline{m}_{x}(\cdot)$, and the control $u_{0}(\cdot)$ of major leader ${\cal A}_{0}$, find an open-loop strategy $\overline{v}_{j}(\cdot)=\overline{v}_{j}[u_{0}(\cdot),\overline{m}_{X}(\cdot),\overline{m}_{x}(\cdot),\xi_{0},\zeta_{j}]\in{\cal V}_{j}^{d}[0,T]$, $1\leq j\leq N_{f}$. On other words, Find the Nash equilibrium response functional $\overline{v}_{j}[\cdot]:{\cal U}_{0}^{d}[0,T]\times L^{2}_{{\cal F}^{0}}(0,T;\mathbb{R}^{n})\times L^{2}_{{\cal F}^{0}}(0,T;\mathbb{R}^{n})\times L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})\times L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})\rightarrow{\cal V}_{j}^{d}[0,T]$ of the following Nash differential games among followers:

[TABLE]

Step 1.2 (CC-F): applying state-aggregation method, it is possible to determine the state-average limit $\overline{m}_{x}$ by the following condition:

[TABLE]

By such step, the Nash equilibrium response functional of follower and $\overline{m}_{x}=\overline{m}_{x}(\xi_{0},\zeta_{j},u_{0},\overline{m}_{X})$ can be specified, given any admissible profile announced by leaders.

Given the approximate Nash response of all followers, we can turn to the Nash analysis of all leaders. To this, it is necessary to have some MM-MFG analysis when there are both major-minor agents.

Step 2: MFG analysis of major-leader: Anticipating the Nash equilibrium response functional of follower $\overline{m}_{x}=\overline{m}_{x}(\xi_{0},\zeta_{j},u_{0},\overline{m}_{X})$, the leaders should solve some Nash equilibrium with size $N_{l}+1$. Similarly, we can assume that as $N_{l}\rightarrow+\infty,$ we suppose $X^{(N_{l})}(\cdot)$ can be approximated by $\mathcal{F}^{0}_{t}$-measurable function $\overline{m}_{X}(\cdot)$. Then the state process of the major leader and minor leaders becomes

[TABLE]

and

[TABLE]

with the following auxiliary cost functionals

[TABLE]

for ${\cal A}_{0}$, and

[TABLE]

for ${\cal A}_{i}^{l},\ 1\leq i\leq N_{l}$. This can be formulated into MM-MFG. We can analyze the optimal control of major leader first. We can set the following auxiliary problem for the major-leader.

Problem (OL2). For given $\xi_{0}\in L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})$ and ${\cal F}^{0}_{t}$-measurable functions $\overline{m}_{X}(\cdot)$, find an open-loop strategy $\overline{u}_{0}(\cdot)\in{\cal U}_{0}^{d}[0,T]$ such that

[TABLE]

Step 3: MFG analysis of minor-leader: Anticipating the Nash equilibrium response functional of follower $\overline{m}_{x}=\overline{m}_{x}(\xi_{0},\zeta_{j},u_{0},\overline{m}_{X})$ and the optimal control $\overline{u}_{0}$ of the major leader. Under the state process (LABEL:G28) with the cost functional (15), we consider the following problem for minor-leaders.

Problem (OL3). For given $\xi_{0},\xi_{i}\in L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})\times L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})$, and ${\cal F}^{0}_{t}$-measurable functions $\overline{m}_{X}(\cdot)$, find an open-loop strategy $\overline{u}_{i}(\cdot)\in{\cal U}_{i}^{d}[0,T]$, $1\leq i\leq N_{l}$, such that

[TABLE]

Step 4: Consistency condition of (Open-loop) Stakelberg-Cornot-Nash equlibrium: CC condition to determine the frozen $\overline{m}_{X}$ by

[TABLE]

And turn to get its global solvability.

In order to show the steps more clearly, here we illustrate the steps by the figure as follows.

(X_{0},u_{0})$$(x_{j},\overline{v}_{j}[u_{0},\overline{m}_{X},\overline{m}_{x}])$$\mathbb{E}[\overline{X}_{i}(\overline{u}_{i}[\overline{X}_{0},\overline{m}_{X}])]=\overline{m}_{X}$$\mathbb{E}[x_{j}(\overline{v}_{j}[u_{0},\overline{m}_{X},\overline{m}_{x}])]=\overline{m}_{x}$$(\overline{X}_{i},\overline{u}_{i}[\overline{X}_{0},\overline{m}_{X}])$$(\overline{X}_{0},\overline{u}_{0}[\overline{m}_{X},\overline{m}_{x}])Step 1.1Step 1.2Step 2Step 3Step 4

3 Open-loop strategies

From now on, we will suppress time variable $t$ in the equation unless it is necessary. In this section, we study the Mixed S-MM-game strategy in OL sense.

3.1 Open-loop strategies for the followers

In this subsection, we solve out Problem (OL1) firstly. The main result of this section can be stated as follows.

Theorem 3.1**.**

Under assumptions (H1), (H2), and let $\zeta_{j}\in L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})$, $u_{0}(\cdot)\in{\cal U}_{0}^{d}[0,T]$, $X_{0}(\cdot)\in L^{2}_{{\cal F}^{0}}(0,T;\mathbb{R}^{n})$, $\overline{m}_{X}(\cdot)$, $\overline{m}_{x}(\cdot)\in L^{2}(0,T;\mathbb{R}^{n})$ be given. Then $\overline{v}_{j}(\cdot)\in{\cal V}_{j}^{d}[0,T]$ is an open-loop decentralized optimal control of Problem (OL1) for initial value $\zeta_{j}$ if and only if the following two conditions hold:

1. (i)

For $j=1,2,\ldots,N_{f}$, the adapted solution $(\overline{x}_{j}(\cdot),\overline{y}_{j}(\cdot),\overline{z}_{j}(\cdot))$ to the FBSDE on $[0,T]$

[TABLE]

satisfies the following stationarity condition:

[TABLE] 2. (ii)

For $j=1,2,\ldots,N_{f}$, the following convexity condition holds:

[TABLE]

where $x_{j}(\cdot)$ is the solution to the FSDE

[TABLE]

Or, equivalently, the map $v_{j}(\cdot)\mapsto J_{j}^{f}(\xi_{0},\zeta_{j},\overline{m}_{X}(\cdot),\overline{m}_{x}(\cdot);u_{0}(\cdot),v_{j}(\cdot))$, $\forall j=1,2,\ldots,N_{f}$ is convex.

Proof. For given $\zeta_{j}\in L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})$, $u_{0}(\cdot)\in{\cal U}_{0}^{d}[0,T]$, $X_{0}(\cdot)\in L^{2}_{{\cal F}^{0}}(0,T;\mathbb{R}^{n})$, $\overline{m}_{X}(\cdot)$, $\overline{m}_{x}(\cdot)\in L^{2}(0,T;\mathbb{R}^{n})$, and $\overline{v}_{j}(\cdot)\in{\cal V}_{j}^{d}[0,T]$, let $(\overline{x}_{j}(\cdot)$, $\overline{y}_{j}(\cdot)$, $\overline{z}_{j}(\cdot))$ be adapted solution to FBSDE (16). For any $v_{j}(\cdot)\in{\cal V}_{j}^{d}[0,T]$ and $\varepsilon\in\mathbb{R}$, let $x_{j}^{\varepsilon}(\cdot)$ be the solution to the following perturbed state equation on $[0,T]$:

[TABLE]

Then denoting $x_{j}(\cdot)$ the solution to the FSDE (LABEL:b2), we have $x_{j}^{\varepsilon}(\cdot)=\overline{x}_{j}(\cdot)+\varepsilon x_{j}(\cdot)$ and

[TABLE]

On the other hand, applying Itô’s formula to \Big{\langle}\overline{y}_{j},x_{j}\Big{\rangle}, and taking expectation, we obtain

[TABLE]

Hence,

[TABLE]

It follows that

[TABLE]

if and only if (17) and (18) hold.

Furthermore, if we assume that $\widetilde{R}$ is invertible, then we have

[TABLE]

so the related Hamiltonian system can be represented by

[TABLE]

Based on above analysis, it follows that

[TABLE]

Here, the first equality of (21) is due to the consistency condition: the frozen term $\overline{m}_{x}(\cdot)$ should equal to the average limit of all realized states $\overline{x}_{j}(\cdot)$; the second equality is due to the law of large numbers. Thus, by replacing $\overline{m}_{x}$ by $\mathbb{E}[\overline{x}_{j}]$, we get the following system

[TABLE]

As all agents are statistically identical, thus we can suppress subscript “$j$” and the following consistency condition system arises for generic agent:

[TABLE]

where $\widetilde{W}$ stands for a generic Brownian motion on $(\Omega,{\cal F},\mathbb{P})$ and it is independent of $W_{0}$. $\zeta$ is a representative element of $\{\zeta_{j}\}_{1\leq j\leq N_{f}}$, and $X_{0}(\cdot)$, $\overline{m}_{X}(\cdot)$ are to be determined.

3.2 Open-loop strategies for the major leader

Once Problem (OL1) is solved, we turn to solve Problem (OL2) about the major leader (agent ${\cal A}_{0}$). Note that when the followers take their optimal respond $\overline{v}_{j}(\cdot)$ given by (20), the major leader ends up with the following state equation system:

[TABLE]

And its cost functional is given by (14). Note that equation (23) is a two-point boundary value problem for SDEs, which is what we call a forward-backward stochastic differential equation (FBSDE; see [26, 33, 32, 34]) and the cost functional is still linear quadratic form. Hence, we are going to solve the LQ problem for a FBSDE. Noting that this FBSDE is coupled, therefore, it is not so easy to deal with it. Let us keep in mind that the “state” for (23) is the triple $(\overline{X}_{0}(\cdot),\overline{x}(\cdot),\overline{y}(\cdot))$. The main result of this section can be stated as follows.

Theorem 3.2**.**

Under assumptions (H1), (H2), and let $\xi_{0},\zeta\in L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})$, $\overline{m}_{X}(\cdot)\in L^{2}(0,T;\mathbb{R}^{n})$ be given. Then $\overline{u}_{0}(\cdot)\in{\cal U}_{0}^{d}[0,T]$ is an open-loop decentralized optimal control of Problem (OL2) for initial value $\xi_{0}$ if and only if the following two conditions hold:

1. (i)

The adapted solution $(\overline{X}_{0}(\cdot),\overline{x}(\cdot),(\overline{y}(\cdot),\overline{z}(\cdot)),(Y_{0}(\cdot),Z_{0}(\cdot)),(p(\cdot),q(\cdot)),K(\cdot))$ to the FBSDE on $[0,T]$

[TABLE]

satisfies the following stationarity condition:

[TABLE] 2. (ii)

The following convexity condition holds:

[TABLE]

where $(X_{0}(\cdot),x(\cdot))$ is the solution to the FBSDE

[TABLE]

Or, equivalently, the map $u_{0}(\cdot)\mapsto J_{0}(\xi_{0},\overline{m}_{X}(\cdot),\overline{m}_{x}(\cdot);u_{0}(\cdot))$ is convex.

Proof. For given $\xi_{0},\zeta\in L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})$, $\overline{m}_{X}(\cdot)\in L^{2}(0,T;\mathbb{R}^{n})$, and $\overline{u}_{0}(\cdot)\in{\cal U}_{0}^{d}[0,T]$, let $(\overline{X}_{0}(\cdot)$, $\overline{x}(\cdot)$, $(\overline{y}(\cdot),\overline{z}(\cdot))$, $(Y_{0}(\cdot),Z_{0}(\cdot))$, $(p(\cdot),q(\cdot))$, $K(\cdot))$ be adapted solution to FBSDE (24). For any $u_{0}(\cdot)\in{\cal U}_{0}^{d}[0,T]$ and $\varepsilon\in\mathbb{R}$, let $X_{0}^{\varepsilon}(\cdot)$, $x^{\varepsilon}(\cdot)$, $(y^{\varepsilon}(\cdot),z^{e}(\cdot))$ be the solution to the following perturbed state equation on $[0,T]$:

[TABLE]

Then denoting $(X_{0}(\cdot),x(\cdot),(y(\cdot),z(\cdot)))$ the solution to the FBSDE (27), we have $X_{0}^{\varepsilon}(\cdot)=\overline{X}_{0}(\cdot)+\varepsilon X_{0}(\cdot)$, $x^{\varepsilon}(\cdot)=\overline{x}(\cdot)+\varepsilon x(\cdot)$, $y^{\varepsilon}(\cdot)=\overline{y}(\cdot)+\varepsilon y(\cdot)$, $z^{\varepsilon}(\cdot)=\overline{z}(\cdot)+\varepsilon z(\cdot)$ and

[TABLE]

On the other hand, applying Itô’s formula to $\langle Y_{0},X_{0}\rangle+\langle p,x\rangle+\langle K,y\rangle$, and taking expectation, we obtain

[TABLE]

Hence,

[TABLE]

It follows that

[TABLE]

if and only if (25) and (LABEL:C6e27) hold.

Similarly, if we assume $R_{0}$ is invertible, then we can represent the optimal control by

[TABLE]

Then the following coupled system follows

[TABLE]

where $\overline{m}_{X}(\cdot)$ is to be determined.

3.3 Open-loop strategies for the minor leaders

Once Problem (OL2) is solved, we turn to solve Problem (OL3) about the minor leaders (agents ${\cal A}_{i}^{l}$, $1\leq i\leq N_{l}$). Note that when the followers takes their optimal responds $\overline{v}_{j}(\cdot)$ given by (20), and the major leader takes his optimal control $\overline{u}_{0}(\cdot)$ given by (28), the minor leaders ends up with the following state equation system:

[TABLE]

And its cost functional is given by (15) with $\overline{X}_{0}(\cdot)$ being from (29). So it is similar to solve Problem (OL1), and the main result in this section can be stated as follows.

Theorem 3.3**.**

Under assumptions (H1), (H2), and let $\xi_{0},\xi_{i}\in L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})$, $\overline{u}_{0}(\cdot)\in{\cal U}_{0}^{d}[0,T]$, $\overline{m}_{X}(\cdot)\in L^{2}(0,T;\mathbb{R}^{n})$ be given. Then $\overline{u}_{i}(\cdot)\in{\cal U}_{i}^{d}[0,T]$ is a decentralized optimal control of Problem (OL3) for initial value $\xi_{i}$ if and only if the following two conditions hold:

1. (i)

For $i=1,2,\ldots,N_{l}$, the adapted solution $(\overline{X}_{i}(\cdot),\overline{Y}_{i}(\cdot),\overline{Z}_{i}(\cdot))$ to the FBSDE on $[0,T]$

[TABLE]

satisfies the following stationarity condition:

[TABLE] 2. (ii)

For $i=1,2,\ldots,N_{l}$, the following convexity condition holds:

[TABLE]

where $X_{i}(\cdot)$ is the solution to the FSDE

[TABLE]

Or, equivalently, the map $u_{i}(\cdot)\mapsto J_{i}^{l}(\xi_{0},\xi_{i},\overline{m}_{X}(\cdot);\overline{u}_{0}(\cdot),u_{i}(\cdot))$ is convex (for $i=1,2,\ldots,N_{l}$).

Proof. For given $\xi_{0},\xi_{i}\in L^{2}_{{\cal F}_{0}}(\Omega;\mathbb{R}^{n})$, $\overline{u}_{0}(\cdot)\in{\cal U}_{0}^{d}[0,T]$, $\overline{m}_{X}(\cdot)\in L^{2}(0,T;\mathbb{R}^{n})$, and $\overline{u}_{i}(\cdot)\in{\cal U}_{i}^{d}[0,T]$, let $(\overline{X}_{i}(\cdot)$, $\overline{Y}_{i}(\cdot)$, $\overline{Z}_{i}(\cdot))$ be adapted solution to FBSDE (30). For any $u_{i}(\cdot)\in{\cal U}_{i}^{d}[0,T]$ and $\varepsilon\in\mathbb{R}$, let $X_{i}^{\varepsilon}(\cdot)$ be the solution to the following perturbed state equation on $[0,T]$:

[TABLE]

Then denoting $X_{i}(\cdot)$ the solution to the FSDE (LABEL:b6), we have $X_{i}^{\varepsilon}(\cdot)=\overline{X}_{i}(\cdot)+\varepsilon X_{i}(\cdot)$ and

[TABLE]

On the other hand, applying Itô’s formula to \Big{\langle}\overline{Y}_{i},X_{i}\Big{\rangle}, and taking expectation, we obtain

[TABLE]

Hence,

[TABLE]

It follows that

[TABLE]

if and only if (31) and (32) hold.

Furthermore, if we assume that $R$ is invertible, then we have

[TABLE]

so the related Hamiltonian system can be represented by

[TABLE]

Based on the above analysis, it follows that

[TABLE]

Here, the first equality of (35) is due to the consistency condition: the frozen term $\overline{m}_{X}(\cdot)$ should equal to the average limit of all realized states $\overline{X}_{i}(\cdot)$; the second equality is due to the law of large numbers. Thus, by replacing $\overline{m}_{X}$ by $\mathbb{E}[\overline{X}_{i}]$, we get the following system

[TABLE]

As all agents are statistically identical, thus we can suppress subscript “$i$” and the following consistency condition system arises for generic agent:

[TABLE]

where $W$ stands for a generic Brownian motion on $(\Omega,{\cal F},\mathbb{P})$, and it is independent of $W_{0},\widetilde{W}$. $\xi$ is a representative element of $\{\xi_{i}\}_{1\leq i\leq N_{l}}$.

To the end of the section, combined with (29) and (LABEL:G4), replacing $\overline{m}_{X}$ by $\mathbb{E}[\overline{X}]$, we can get the consistency condition system for open-loop strategy as follows.

[TABLE]

4 The Consistency Condition System

Under assumptions (H1), (H2), when $\widetilde{R}(\cdot)$, $R_{0}(\cdot)$ and $R(\cdot)$ are always invertible, we get the consistency condition (CC) for OL strategy in section 3. In this section, we turn to verify the well-posedness of the CC equation.

For the simplicity of notation, denote $\mathbb{X}^{\top}=(\overline{X}_{0},\overline{X},\overline{x},K)$, $\mathbb{Y}^{\top}=(Y_{0},\overline{Y},\overline{y},p)$, $\mathbb{Z}^{\top}=(Z_{0},\overline{Z},\overline{z},q)$, $\textbf{W}^{\top}=(W_{0},W,\widetilde{W},\widetilde{W})$, and then the consistency condition system (LABEL:G5) can be rewritten as

[TABLE]

where

[TABLE]

4.1 Decoupling for open-loop strategy

Then, we turn to decouple the FBSDE (38) by Riccati equation. Note that

[TABLE]

Hence,

[TABLE]

Now, we assume that

[TABLE]

for some deterministic and differentiable functions $P(\cdot)$ and $\Pi(\cdot)$, taking values in ${\cal S}^{4n}$, such that

[TABLE]

Then

[TABLE]

and

[TABLE]

Therefore,

[TABLE]

Comparing the diffusion terms, we should have

[TABLE]

Then

[TABLE]

and

[TABLE]

Comparing the drift terms, we should have

[TABLE]

Therefore, we should let $P(\cdot)$ and $\Pi(\cdot)$ be the solutions to the following Riccati equations, respectively:

[TABLE]

and

[TABLE]

4.2 Decoupling for the feedback strategy

Except the pure open-loop method, we can also introducing the following Riccati equations to decouple the Hamiltonian systems first.

The Hamiltonian system of minor follower is

[TABLE]

with the stationary condition

[TABLE]

Assume that $\overline{y}_{j}=P_{1}\overline{x}_{j}+\Phi_{1}$, and we can get the Riccati equations

[TABLE]

and

[TABLE]

where

[TABLE]

Note that

[TABLE]

so the feedback is

[TABLE]

The major leader ends up with the following Hamiltonian system

[TABLE]

with the stationary condition

[TABLE]

where

[TABLE]

Assume that $\mathchoice{\overarrow@\arrowfill@\relbar\relbar\rightharpoonup\displaystyle{Y}}{\overarrow@\arrowfill@\relbar\relbar\rightharpoonup\textstyle{Y}}{\overarrow@\arrowfill@\relbar\relbar\rightharpoonup\scriptstyle{Y}}{\overarrow@\arrowfill@\relbar\relbar\rightharpoonup\scriptscriptstyle{Y}}=P_{2}\mathchoice{\overarrow@\arrowfill@\relbar\relbar\rightharpoonup\displaystyle{X}}{\overarrow@\arrowfill@\relbar\relbar\rightharpoonup\textstyle{X}}{\overarrow@\arrowfill@\relbar\relbar\rightharpoonup\scriptstyle{X}}{\overarrow@\arrowfill@\relbar\relbar\rightharpoonup\scriptscriptstyle{X}}+\Phi_{2}$, where

[TABLE]

and for simplicity, we rewrite the Hamiltonian system by

[TABLE]

where

[TABLE]

then we can get the following Riccati equations

[TABLE]

and

[TABLE]

So the feedback is

[TABLE]

At last, the Hamiltonian system of minor leader is

[TABLE]

with the stationary condition

[TABLE]

Assume that $\overline{Y}_{i}=P_{3}\overline{X}_{i}+\Phi_{3}$, and we can get the Riccati equations

[TABLE]

and

[TABLE]

where

[TABLE]

Note that

[TABLE]

so the feedback is

[TABLE]

5 $\varepsilon$-Nash Equilibrium Analysis

In above sections, we obtained the decentralized open-loop strategy of the mixed S-MM-MFG through the consistency condition system. Now we turn to verify that it is the SNC approximate equilibrium (i.e. $\varepsilon$-Stackelberg-Nash-Cournot equilibrium). In order to ensure the solvability of the open-loop strategy, we assume the Riccati equation (LABEL:C6e18) and (LABEL:C6e19) admits a unique solution. At the beginning, we first present the definition of $\varepsilon$-SNC equilibrium.

Definition 5.1**.**

A set of controls $(\overline{u}_{0},\overline{u}_{1},\ldots,\overline{u}_{N_{l}},\overline{v}_{1},\ldots,\overline{v}_{N_{f}})\in{\cal U}_{0}^{d}\times{\cal U}^{d}\times{\cal V}^{d}$, for $(1+N_{l}+N_{f})$ agents is called to satisfy an $\varepsilon$-SNC equilibrium with respect to the costs $({\cal J}_{0},{\cal J}_{1}^{l},\ldots,{\cal J}_{N_{l}}^{l},{\cal J}_{1}^{f},{\cal J}_{N_{f}}^{f})$, if there exists $\varepsilon=\varepsilon(N)\geq 0$ ($N=\min\{N_{l},N_{f}\}$), $\lim_{N\rightarrow\infty}\varepsilon(N)=0$ such that for any fixed $i=1,2,\ldots,N_{l}$, $j=1,2,\ldots,N_{f}$, we have

[TABLE]

when any alternative control $(u_{0},u_{i},v_{j})\in{\cal U}_{0}^{d}\times{\cal U}_{i}^{d}\times{\cal V}_{j}^{d}$ is applied by $({\cal A}_{0},{\cal A}_{i},{\cal B}_{j})$.

At first, we present the main result in this section and its proof will be given later.

Theorem 5.1**.**

Under assumptions (H1)-(H2), and if the conditions in the Theorem 3.1, Theorem 3.2, Theorem 3.3 hold, then $(\overline{u}_{0},\overline{u}_{i},\overline{v}_{j})$ is an $\varepsilon$-Nash equilibrium of mixed S-MM-MFG for major leader agent ${\cal A}_{0}$, each minor leader agent ${\cal A}_{i}^{l}$, $i=1,2,\ldots,N_{l}$, and each follower agent ${\cal A}_{j}^{f}$, $j=1,2,\ldots,N_{f}$. And $(\overline{u}_{0},\overline{u}_{i},\overline{v}_{j})$ is given by

[TABLE]

for $(Y_{0},Z_{0})$, $(\overline{Y}_{i},\overline{Z}_{i})$, $(\overline{y}_{j},\overline{z}_{j})$ solved by (LABEL:G5).

For major leader ${\cal A}_{0}$, minor leaders ${\cal A}_{i}^{l}$ and followers ${\cal A}_{j}^{f}$, the decentralized states $\overline{X}_{0}$, $\overline{X}_{i}$ and $\overline{Y}_{j}$ are given respectively by

[TABLE]

where the processes $(Y_{0},Z_{0})$, $(\overline{Y}_{i},\overline{Z}_{i})$, $(\overline{y}_{j},\overline{z}_{j})$ solved by (LABEL:G5). Let us first present following several lemmas and for the simplicity of notation, we denote the inner product $\langle\ \cdot\ ,\ \cdot\ \rangle=|\ \cdot\ |^{2}$.

Lemma 5.1**.**

Under assumptions (H1)-(H2), and if the conditions in the Theorem 3.1, Theorem 3.2, Theorem 3.3 hold, then there exists a constant $M$ independent of $N_{l}$ and $N_{f}$, such that

[TABLE]

[TABLE]

Proof. From Theorem 3.1, Theorem 3.2, Theorem 3.3, the FBSDEs (16), (24) and (30) have a unique solution $(\overline{X}_{0},Y_{0},Z_{0})\in L^{2}_{{\cal F}^{0}}(0,T;\mathbb{R}^{3n})$, $(\overline{X}_{i},\overline{Y}_{i},\overline{Z}_{i})\in L^{2}_{{\cal F}^{i}}(0,T;\mathbb{R}^{3n})$ and $(\overline{x}_{j},\overline{y}_{j},\overline{z}_{j})\in L^{2}_{{\cal G}^{j}}(0,T;\mathbb{R}^{3n})$, $1\leq i\leq N_{l}$, $1\leq j\leq N_{f}$. Thus, SDEs system (47) has also a unique solution

[TABLE]

From (47), by using Burkholder-Davis-Gundy (BDG) inequality, there exists a constant $M$, independent of $N_{l}$ and $N_{f}$, such that for any $t\in[0,T]$,

[TABLE]

and by Gronwall’s inequality, we obtain

[TABLE]

Similarly, we have

[TABLE]

and

[TABLE]

Thus

[TABLE]

and

[TABLE]

By Gronwall’s inequality, it follows that \mathbb{E}\Big{[}\sup_{0\leq s\leq t}\sum_{i=1}^{N_{l}}\big{|}\overline{X}_{i}(s)\big{|}^{2}\Big{]}=O(N_{l}), and \mathbb{E}\Big{[}\sup_{0\leq s\leq t}\sum_{j=1}^{N_{f}}\big{|}\overline{x}_{j}(s)\big{|}^{2}\Big{]}=O(N_{f}). Then, substituting this estimate to (49) and (50) and Gronwall’s inequality yields \mathbb{E}\Big{[}\sup_{0\leq s\leq t}\big{|}\overline{X}_{i}(s)\big{|}^{2}\Big{]}\leq M, and \mathbb{E}\Big{[}\sup_{0\leq s\leq t}\big{|}\overline{x}_{j}(s)\big{|}^{2}\Big{]}\leq M. By applying this estimate to (48), we get \mathbb{E}\Big{[}\sup_{0\leq s\leq t}\big{|}\overline{X}_{0}(s)\big{|}^{2}\Big{]}\leq M.

Now, we recall that

[TABLE]

then we have

Lemma 5.2**.**

Under assumptions (H1)-(H2), and if the conditions in the Theorem 3.1, Theorem 3.2, Theorem 3.3 hold, then there exists a constant $M$ independent of $N_{l}$ and $N_{f}$, such that

[TABLE]

[TABLE]

Proof. For the first one, we have

[TABLE]

From (LABEL:N2), by using Burkholder-Davis-Gundy (BDG) inequality and Lemma 5.1, there exists a constant $M$, independent of $N_{l}$ and $N_{f}$, such that for any $t\in[0,T]$,

[TABLE]

and by Gronwall’s inequality, we obtain

[TABLE]

By the same way, we can prove the second formula.

Lemma 5.3**.**

Under assumptions (H1)-(H2), and if the conditions in the Theorem 3.1, Theorem 3.2, Theorem 3.3 hold, then there exists a constant $M$ independent of $N_{l}$ and $N_{f}$, we have

[TABLE]

where $N:=\min\{N_{l},N_{f}\}$.

Proof. Let us first consider the major leader agent. Recall (4) and (14), we have

[TABLE]

By Hölder inequality and Lemma 5.1, there exists a constant $M$ independent of $N_{l}$ and $N_{f}$ such that

[TABLE]

Noting (52), (53) and Lemma 5.2, there exists a constant $M$ independent of $N_{l}$ and $N_{f}$ such that

[TABLE]

The rest two claims can be proved in the same way.

Remark 5.1**.**

We denote $M$ the common constant of the different boundaries. In the above lemmas, the constant $M$ may vary each line by line but it is always independent of the number of minor-leader agents $N_{l}$ and the number of follower agents $N_{f}$.

5.1 Major leader agent’s perturbation

In this subsection, we will prove that the control strategies set $(\overline{u}_{0},\overline{u}_{1},\ldots,\overline{u}_{N_{l}},\overline{v}_{1},\ldots,\overline{v}_{N_{f}})$ given by Theorem 5.1 is an $\varepsilon$-Nash equilibrium of mixed S-MM-MFG for major leader agent, i.e. there exists an $\varepsilon=\varepsilon(N)\geq 0$, $\lim_{N\rightarrow\infty}\varepsilon(N)=0$ such that

[TABLE]

Let us consider that the major leader agent ${\cal A}_{0}$ uses an alternative strategy $u_{0}$, each minor leader agent ${\cal A}_{i}^{l}$ uses the control $\overline{u}_{i}=-R^{-1}(t)(B(t)^{\top}\overline{Y}_{i}(t)+D(t)^{\top}\overline{Z}_{i}(t))$ and each follower agent ${\cal A}_{j}^{f}$ uses the control $\overline{v}_{j}=-\widetilde{R}^{-1}(t)(\widetilde{B}(t)^{\top}\overline{y}_{j}(t)+\widetilde{D}(t)^{\top}\overline{z}_{j}(t))$. To prove $(\overline{u}_{0},\overline{u}_{1},\ldots,\overline{u}_{N_{l}},\overline{v}_{1},\ldots,\overline{v}_{N_{f}})$ is an $\varepsilon$-Nash equilibrium for the major leader agent, we need to show that for possible alternative control $u_{0}$, $\inf_{u_{0}\in{\cal U}_{0}^{c}[0,T]}{\cal J}_{0}(u_{0},\overline{u},v[u_{0}])\geq{\cal J}_{0}(\overline{u}_{0},\overline{u},\overline{v})-\varepsilon$. Then we only need to consider the perturbation $u_{0}\in{\cal U}_{0}^{c}[0,T]$ such that ${\cal J}_{0}(u_{0},\overline{u},v[u_{0}])\leq{\cal J}_{0}(\overline{u}_{0},\overline{u},\overline{v})$. By the representation of the cost functional in [35, 31], we can give the representation of the cost functional as follows.

Proposition 5.1**.**

Let (H1)-(H2) hold. There exists a bounded self-adjoint linear operator $N_{0}:{\cal U}_{0}^{c}[0,T]\rightarrow{\cal U}_{0}^{c}[0,T]$, a bounded linear operator $H_{0}:\mathbb{R}^{n}\rightarrow{\cal U}_{0}^{c}[0,T]$, a bounded real-valued function $M_{0}:\mathbb{R}^{n}\rightarrow\mathbb{R}$ such that

[TABLE]

Proof. Refer to Proposition 3.1 in [31].

So if we assume that $N_{0}\gg 0$, from Lemma 5.3, then there exists a bounded constant $c$, such that

[TABLE]

which implies that \mathbb{E}\Big{[}\int_{0}^{T}\big{|}u_{0}(t)\big{|}^{2}\mathrm{d}t\Big{]}\leq M, where $M$ is a constant independent of $N$. In fact, by bounded inverse theorem, $N_{0}^{-1}$ is bounded, so there exists a constant $0<\gamma\leq\|N_{0}^{\frac{1}{2}}\|$, such that

[TABLE]

Then we have \mathbb{E}\Big{[}\int_{0}^{T}\big{|}u_{0}(t)\big{|}^{2}\mathrm{d}t\Big{]}\leq M. Similar to Lemma 5.1, we can show that

[TABLE]

Lemma 5.4**.**

Under assumptions (H1)-(H2), and if the conditions in the Theorem 3.1, Theorem 3.2, Theorem 3.3 hold, for the major leader agent’s perturbation control $u_{0}$, we have

[TABLE]

Proof. Recall (4) and (14), we have

[TABLE]

By Hölder inequality and (55), there exists a constant $M$ independent of $N_{l}$ and $N_{f}$ such that

[TABLE]

At last, same as the Lemma 5.3, noting (56), (57), and Lemma 5.2, there exists a constant $M$ independent of $N_{l}$ and $N_{f}$ such that

[TABLE]

Taking the advantage of Lemma 5.3 and Lemma 5.4, we can give the first part of the proof to the Theorem 5.1, i.e. the control strategies set $(\overline{u}_{0},\overline{u}_{1},\ldots,\overline{u}_{N_{l}},\overline{v}_{1},\ldots,\overline{v}_{N_{f}})$ given by Theorem 5.1 is an $\varepsilon$-Nash equilibrium of the mixed S-MM-MFG for major leader agent.

Part A of the proof to Theorem 5.1

Combining Lemma 5.3 and Lemma 5.4, we have

[TABLE]

where the second inequality comes from the fact that $J_{0}(\overline{u}_{0})=\inf_{u_{0}\in{\cal U}_{0}^{c}[0,T]}J_{0}(u_{0})$. Consequently, the Theorem 5.1 holds for the major leader agent with \varepsilon=O\Big{(}\frac{1}{\sqrt{N}}\Big{)}.

5.2 Minor leader agent’s perturbation

Now, let us consider the following case: a given minor leader agent ${\cal A}_{i}^{l}$ uses an alternative strategy $u_{i}\in{\cal U}_{i}^{c}[0,T]$, the major leader agent uses $\overline{u}_{0}$, each follower agent ${\cal A}_{j}^{f}$ uses $\overline{v}_{j}$ while other minor leader agents use the control $\overline{u}_{-i}$. In fact, by the representation of the cost functional (which is similar to the argument of major leader agent), to prove $(\overline{u}_{0},\overline{u}_{1},\ldots,\overline{u}_{N_{l}},\overline{v}_{1},\ldots,\overline{v}_{N_{f}})$ is an $\varepsilon$-Nash equilibrium for the minor leader agent, we only need to consider the perturbation $u_{i}\in{\cal U}_{i}^{c}[0,T]$ satisfying

[TABLE]

where $M$ is a constant independent of $N_{l}$. Then similar to Lemma 5.1, we can show that

[TABLE]

Lemma 5.5**.**

Under assumptions (H1)-(H2), and if the conditions in the Theorem 3.1, Theorem 3.2, Theorem 3.3 hold, then there exists a constant $M$ independent of $N_{l}$ and $N_{f}$, such that

[TABLE]

where X^{(i,N_{l})}(t)=\frac{1}{N_{l}}\big{(}X_{i}(t)+\sum_{k\neq i}\overline{X}_{k}(t)\big{)}.

Proof. In fact, we have

[TABLE]

by (59) , it yields

[TABLE]

Combined with Lemma 5.2, we can directly get

[TABLE]

Lemma 5.6**.**

Under assumptions (H1)-(H2), and if the conditions in the Theorem 3.1, Theorem 3.2, Theorem 3.3 hold, for the minor leader agent’s perturbation control $u_{i}$, we have

[TABLE]

Proof. Recall (5) and (15), we have

[TABLE]

By same technique, applying Hölder inequality, Lemma 5.5, and (59), there exists a constant $M$ independent of $N_{l}$ and $N_{f}$ such that

[TABLE]

Taking the advantage of Lemma 5.3 and Lemma 5.6, we can give the second part of the proof to the Theorem 5.1, i.e. the control strategies set $(\overline{u}_{0},\overline{u}_{1},\ldots,\overline{u}_{N_{l}},\overline{v}_{1},\ldots,\overline{v}_{N_{f}})$ given by Theorem 5.1 is an $\varepsilon$-Nash equilibrium of the mixed S-MM-MFG for minor leader agent.

Part B of the proof to Theorem 5.1

Combining Lemma 5.3 and Lemma 5.6, we have

[TABLE]

where the second inequality comes from the fact that $J_{i}^{l}(\overline{u}_{0},\overline{u}_{i})=\inf_{u_{i}\in{\cal U}_{i}^{c}[0,T]}J_{i}^{l}(\overline{u}_{0},u_{i})$. Consequently, the Theorem 5.1 holds for the minor leader agent with \varepsilon=O\Big{(}\frac{1}{\sqrt{N}}\Big{)}.

5.3 Follower agent’s perturbation

At last, we consider the following case: a given follower agent ${\cal A}_{j}^{f}$ uses an alternative strategy $v_{j}\in{\cal V}_{j}^{c}[0,T]$, the major leader agent uses $\overline{u}_{0}$, each minor leader agent ${\cal A}_{i}^{l}$ uses $\overline{u}_{i}$ while other follower agents use the control $\overline{v}_{-j}$. In fact, by the representation of the cost functional (which is similar to the argument of major leader agent), to prove $(\overline{u}_{0},\overline{u}_{1},\ldots,\overline{u}_{N_{l}},\overline{v}_{1},\ldots,\overline{v}_{N_{f}})$ is an $\varepsilon$-Nash equilibrium for the follower agent, we only need to consider the perturbation $v_{j}\in{\cal V}_{j}^{c}[0,T]$ satisfying

[TABLE]

where $M$ is a constant independent of $N$. Then similar to Lemma 5.1, we can show that

[TABLE]

Lemma 5.7**.**

Under assumptions (H1)-(H2), and if the conditions in the Theorem 3.1, Theorem 3.2, Theorem 3.3 hold, then there exists a constant $M$ independent of $N_{l}$ and $N_{f}$, such that

[TABLE]

where x^{(j,N_{f})}(t)=\frac{1}{N_{f}}\big{(}x_{j}(t)+\sum_{k\neq j}\overline{x}_{k}(t)\big{)}.

Proof. In fact, we have

[TABLE]

by (62) , it yields

[TABLE]

Combined with Lemma 5.2, we can directly get

[TABLE]

Lemma 5.8**.**

Under assumptions (H1)-(H2), and if the conditions in the Theorem 3.1, Theorem 3.2, Theorem 3.3 hold, for the follower agent’s perturbation control $v_{j}$, we have

[TABLE]

Proof. Recall (6) and (11), we have

[TABLE]

By Hölder inequality and (62) there exists a constant $M$ independent of $N_{l}$ and $N_{f}$ such that

[TABLE]

At last, same as the Lemma 5.3, noting (63), (64), and Lemma 5.7, there exists a constant $M$ independent of $N_{l}$ and $N_{f}$ such that

[TABLE]

Taking the advantage of Lemma 5.3 and Lemma 5.8, we can give the last part of the proof to the Theorem 5.1, i.e. the control strategies set $(\overline{u}_{0},\overline{u}_{1},\ldots,\overline{u}_{N_{l}},\overline{v}_{1},\ldots,\overline{v}_{N_{f}})$ given by Theorem 5.1 is an $\varepsilon$-Nash equilibrium of the mixed S-MM-MFG for follower agent.

Part C of the proof to Theorem 5.1

Combining Lemma 5.3 and Lemma 5.8, we have

[TABLE]

where the second inequality comes from the fact that $J_{j}^{f}(\overline{u}_{0},\overline{v}_{j})=\inf_{v_{j}\in{\cal V}_{j}^{c}[0,T]}J_{j}^{f}(\overline{u}_{0},v_{j})$. Consequently, the Theorem 5.1 holds for the follower agent with \varepsilon=O\Big{(}\frac{1}{\sqrt{N}}\Big{)}. Finally, combined with the Part A, Part B, we complete the proof to Theorem 5.1.

6 Special Case

In this section, we will give an example to show how the major leader influences the whole system. We now look at a special case in which the major leader does not appear. In this case the problem is reduced to a leader-follower mean-field LQG game problem. Let us still regard it as if the major leader does appear but does not affect the game at all, i.e., we assume that

[TABLE]

Moreover, let $\lambda=1$, $\widetilde{\lambda}_{1}=0$, $\widetilde{\lambda}_{2}=\widetilde{\lambda}$, and $\widetilde{\lambda}_{3}=1-\widetilde{\lambda}$. By observation, we can find that the coupled mean-field term between the leaders and the followers appear on the cost functional of the followers. Thus, it is truly a leader-follower mean-field LQG game problem. By the analysis above, we can get the CC equation of the special case as follows.

[TABLE]

Furthermore, we find that the equation of $K(\cdot)$ and $(p(\cdot),q(\cdot))$ is coupled together but decoupled with other equations and if the consistency condition equation admits a unique adapted solution then $K\equiv 0$, $p\equiv 0$ and $q\equiv 0$ is the trivial solution to the equation. Then the CC equation of the special case is simplified as

[TABLE]

Next, by the decoupling for the open-loop strategy, we get the Riccati equations (LABEL:C6e18) and (LABEL:C6e19). However, it is still hard to get the explicit solution to the Riccati equations. So we consider the $1$-dimensional example 5.1 as follows.

Example 5.1 Let $n=m_{1}=m_{2}=m_{3}=1$, (66) holds, and let

[TABLE]

we have the state equation

[TABLE]

The cost functional reads

[TABLE]

and

[TABLE]

and the Riccati equation

[TABLE]

where

[TABLE]

so it can be easily solved out that

[TABLE]

And the optimal control $\overline{u}_{i}=-\overline{Z}=0$, $\overline{v}_{j}=-\overline{z}=0$, subject to the optimal cost functional ${\cal J}_{i}^{l}=\frac{1}{2}\xi_{i}^{2}$, and ${\cal J}_{j}^{f}=\frac{1}{2}\zeta_{j}^{2}$.