A second-order stochastic maximum principle for generalized mean-field control problem

TL;DR

This paper develops a second-order stochastic maximum principle for generalized mean-field control problems where the standard SMP fails due to Hamiltonian singularity, using advanced backward stochastic differential equations.

Contribution

It introduces a second-order SMP for mean-field control problems with singular Hamiltonians, expanding the theoretical framework beyond first-order conditions.

Findings

Derived a second-order SMP applicable to singular Hamiltonian cases

Introduced a generalized mean-field backward stochastic differential equation for adjoint processes

Provided mathematical tools for handling singularities in stochastic control

Abstract

In this paper, we study the generalized mean-field stochastic control problem when the usual stochastic maximum principle (SMP) is not applicable due to the singularity of the Hamiltonian function. In this case, we derive a second order SMP. We introduce the adjoint process by the generalized mean-field backward stochastic differential equation. The keys in the proofs are the expansion of the cost functional in terms of a perturbation parameter, and the use of the range theorem for vector-valued measures.

Equations

$$\left\{\begin{array}[]{l}dX_{t}=b(t,X_{t},P_{{X_{t}}},v_{t})dt+\sigma(t,X_{t},P_{{X_{t}}})dB_{t},\\ X_{0}=x,\end{array}\right.$$

$$\begin{array}{r}\begin{array}{r}J(v)=\mathbb{E}\left\{{\int }_0^{T}h(t,X_t,P_{X_t},v_t)dt+\Phi (X_T,P_{X_T})\right\},\end{array}\end{array}$$

$$J(u)=\underset{v\in \mathcal{U}}{\min }J(v).$$

$$\left\{\begin{array}[]{ccl}dX^{v}_{t}&=&v_{t}dt+\left\{(X^{v}_{t}-1)+\mathbb{E}\left[(X^{v}_{t}-1)\right]\right\}dB_{t},\ v\in{U}:=\{-1,0,1\},\\ X^{v}_{0}&=&1,\end{array}\right.$$

$$J(v)=\frac{1}{2}\mathbb{E}\big{\{}(X^{v}_{T}-1)+\tilde{\mathbb{E}}\big{[}(X^{v}_{T}-1)\big{]}\big{\}}^{2}.$$

$$\left\{\begin{array}[]{ccl}dp_{t}&=&\left\{q_{t}+\mathbb{E}\left[{q_{t}}\right]\right\}dt-q_{t}dB_{t}\\ p_{T}&=&0.\end{array}\right.$$

$$H(t,X_t^u,P_{X_t^u},p_t,q_t,v)\equiv 0,v\in U.$$

$$H(t,X_t^{\tilde{u}},P_{X_t^{\tilde{u}}},p_t^{\tilde{u}},q_t^{\tilde{u}},{\tilde{u}}_t)=H(t,X_t^{\tilde{u}},P_{X_t^{\tilde{u}}},p_t^{\tilde{u}},q_t^{\tilde{u}},v),a.s.$$

$$\left\{\begin{array}[]{l}dX_{t}=b(t,X_{t},v_{t})dt+\sigma(t,X_{t})dB_{t},\\ X_{0}=x.\end{array}\right.$$

$$\begin{array}{r}\begin{array}{r}J(v)=\mathbb{E}\{{\int }_0^{T}h(t,X_t,v_t)dt+\Phi (X_T)\},\end{array}\end{array}$$

$$W_2(P_{\mu },P_{\nu })=\inf \left\{{\left(\mathbb{E}[|{\mu }^{\prime }-{\nu }^{\prime }{|}^2]\right)}^{\frac{1}{2}}\right\},$$

$$\tilde{f}({\mathfrak{v}}_0+\eta )-\tilde{f}({\mathfrak{v}}_0)=D\tilde{f}({\mathfrak{v}}_0)(\eta )+o(|\eta {|}_{L^2}),$$

$$f(P_{\mathfrak{v}})-f(P_{{\mathfrak{v}}_0})=\mathbb{E}\left[h_0({\mathfrak{v}}_0)(\mathfrak{v}-{\mathfrak{v}}_0)\right]+o(|\mathfrak{v}-{\mathfrak{v}}_0{|}_{L^2}),\mathfrak{v}\in L^2(\mathcal{F};{\mathbb{R}}^n).$$

$$\begin{array}{r}\begin{array}{rlr}i)& |{\partial }_{\mu }f(\mu ,x)|\le C,\forall \mu \in {\mathcal{P}}_2({\mathbb{R}}^d),x\in {\mathbb{R}}^d,& \\ ii)& |{\partial }_{\mu }f(\mu ,x)-{\partial }_{\mu }f({\mu }^{\prime },x^{\prime })|\le C\left(W_2(\mu ,{\mu }^{\prime })+|x-x^{\prime }|\right),& \\ & \forall \mu ,{\mu }^{\prime }\in {\mathcal{P}}_2({\mathbb{R}}^d),x,x^{\prime }\in {\mathbb{R}}^d;& \end{array}\end{array}$$

$${\partial }_{\mu }^{2}f(\mu ,x,y):={\left({\partial }_{\mu }\left(\left({\partial }_{\mu }f{)}_j\left(\cdot ,y\right)\right)(\mu ,x\right)\right)}_{1\le j\le d},(\mu ,x,y)\in {\mathcal{P}}_2({\mathbb{R}}^d)\times {\mathbb{R}}^d\times {\mathbb{R}}^d,$$

$$\tilde{f}({\mathfrak{v}}_0+\eta )-\tilde{f}({\mathfrak{v}}_0)$$

$${\partial }_{\mu }f(P_{{\mathfrak{v}}_0},y)=g^{\prime }(\mathbb{E}[h({\mathfrak{v}}_0)])({\partial }_{y}h)(y),y\in {\mathbb{R}}^d.$$

$${\partial }_{\mu }^{2}f(P_{{\mathfrak{v}}_0},x,y)=g^{\prime \prime }(\mathbb{E}[h({\mathfrak{v}}_0)])({\partial }_{x}h)(x)\times ({\partial }_{y}h)(y),$$

$${\partial }_y{\partial }_{\mu }f(P_{{\mathfrak{v}}_0},y)=g^{\prime }(\mathbb{E}[h({\mathfrak{v}}_0)])({\partial }_y^{2}h)(y).$$

$$\begin{array}{r}\begin{array}{r}{X}_s^{t,\xi }=\xi +{\int }_t^s\sigma (X_r^{t,\xi },P_{X_r^{t,\xi }})d{B}_r+{\int }_t^s\sigma (X_r^{t,\xi },P_{X_r^{t,\xi }})dr,s\in [t,T],\end{array}\end{array}$$

$$\begin{array}{r}\begin{array}{r}{X}_s^{t,x,\xi }=x+{\int }_t^s\sigma (X_r^{t,x,\xi },P_{X_r^{t,\xi }})d{B}_r+{\int }_t^s\sigma (X_r^{t,x,\xi },P_{X_r^{t,\xi }})dr,s\in [t,T].\end{array}\end{array}$$

$$\Phi (X_s^{t,x,P_{\xi }},P_{X_s^{t,\xi }})-\Phi (x,P_{\xi })$$

$$b(t):=b(t,X_t^u,P_{X_t^u},u_t),b_x(t):=b_x(t,X_t^u,P_{X_t^u},u_t),$$

$$\tilde{b}(t):=b(t,{\tilde{X}}_t^u,P_{{\tilde{X}}_t^u},{\tilde{u}}_t),{\tilde{b}}_x(t):=b_x(t,{\tilde{X}}_t^u,P_{{\tilde{X}}_t^u},{\tilde{u}}_t),$$

$$b_{xx}(t):=b_{xx}(t,X_t^u,P_{X_t^u},u_t),b_{\mu }(t):=b_{\mu }(t,X_t^u,P_{X_t^u},{\tilde{X}}_t^u,u_t),$$

$${\tilde{b}}_{\mu }(t):=b_{\mu }(t,{\tilde{X}}_t^u,P_{{\tilde{X}}_t^u},X_t^u,{\tilde{u}}_t),b_{\mu \mu }(t):=b_{\mu \mu }(t,X_t^u,P_{X_t^u},{\tilde{X}}_t^u,{\bar{X}}_t^u,u_t),$$

$${\widetilde{\stackrel{ˉ}{b}}}_{\mu \mu }(t):=b_{\mu \mu }(t,{\tilde{X}}_t^u,P_{{\tilde{X}}_t^u},{\bar{X}}_t^u,X_t^u,{\tilde{u}}_t),b_{x\mu }(t):=b_{x\mu }(t,X_t^u,P_{X_t^u},{\tilde{X}}_t^u,u_t),$$

$$b_{y\mu }(t):=b_{y\mu }(t,X_t^u,P_{X_t^u},{\tilde{X}}_t^u,u_t),△b(t;v):=b(t,X_t^u,P_{X_t^u},v)-b(t),$$

$$△b_x(t;v):=b_x(t,X_t^u,P_{X_t^u},v)-b_x(t),△b_{\mu }(t;v):=b_{\mu }(t,X_t^u,P_{X_t^u},{\tilde{X}}_t^u,v)-b_{\mu }(t),$$

$$\left\{\begin{array}[]{rcl}-dp_{t}&=&\bigg{\{}b_{x}(t)p_{t}+\sigma_{x}(t)q_{t}+h_{x}(t)\\ &&+\tilde{\mathbb{E}}\Big{[}\tilde{b}_{\mu}(t)\tilde{p}_{t}+\tilde{\sigma}_{\mu}(t)\tilde{q}_{t}+\tilde{h}_{\mu}(t)\Big{]}\bigg{\}}dt-q_{t}dB_{t},\\ p_{T}&=&\Phi_{x}(X^{u}_{T},P_{X^{u}_{T}})+\tilde{\mathbb{E}}\left[\Phi_{\mu}(\tilde{X}^{u}_{T},P_{\tilde{X}^{u}_{T}},X^{u}_{T})\right].\end{array}\right.$$

Full text

Research supported partially by FDCT 025/2016/A1.

∗ Corresponding author: Hancheng Guo

Hancheng Guo∗

Department of Mathematics, Faculty of Science and Technology

University of Macau

Macau, 999078, P. R. China

Jie Xiong

Department of Mathematics, Faculty of Science and Technology

University of Macau

Macau, 999078, P. R. China

(Communicated by the associate editor name)

1. Introduction

We consider the following optimal stochastic control problem of mean-field type with the state equation

[TABLE]

and the cost functional

[TABLE]

where $P_{\xi}$ denotes the law of the random variable $\xi$.

The agent wishes to minimize his cost functional, namely, an admissible control $u\in\mathcal{U}$ is said to be optimal if

[TABLE]

where $\mathcal{U}$ is the set of all admissible controls to be defined later in Section 3.

About stochastic maximum principle (SMP), some pioneering works have been done by Pontryagin et al. [22]. They obtained Pontryagin’s maximum principle by using “spike variation”. Kushner ([14], [15]) studied the SMP in the framework when the diffusion coefficient does not depend on the control variable, and the cost functional consists of terminal cost only. Haussmann [10] gave a version of SMP when the diffusion of the state does not depend on the control variable. Arkin and Saksonov [1], Bensoussan [3] and Bismut [4], proved different versions of SMP under various setups. An SMP was obtained by Peng [21] in 1990. In that paper, first and second order variational inequalities are introduced, when the control domain need not to be convex, and the diffusion coefficient contains the control variable.

Pardoux and Peng [20] introduced non-linear backward stochastic differential equations (BSDE) in 1990. They showed that under appropriate assumptions, BSDE admits an unique adapted solution, and the associated comparison theorem holds. Buckdahn et al [6] obtained mean-field BSDE in a natural way as the limit of some high dimensional system of forward and backward stochastic differential equations. Li [16] studied SMP for mean-filed controls when the domain of the control is assumed to be convex. Under some additional assumptions, both necessary and sufficient conditions for the optimality of a control were proved. Buckdahn et al [7] studied generalized mean-field stochastic differential equations and the associated partial differential equations (PDEs). “Generalized” means that the coefficients depend on both the state process and its law. They proved that under appropriate regularity conditions on the coefficients, the SDE has a unique classical solution. Buckdahn et al. [5] obtained SMP for generalized mean-field system in 2016.

Sometimes, the Hamiltonian function becomes constant in the control variable, as we will see in the next example, which makes the aforementioned SMP not applicable.

Example 1.1**.**

Consider the control problem with state equation:

[TABLE]

and cost functional:

[TABLE]

For the control $u_{t}\equiv 0$, $X^{u}_{t}\equiv 1$ is the unique solution of (3). It is clear that $J(u)=0$, and hence, $u$ is an optimal control. On the other hand, the first order adjoint processes satisfy the following equation:

[TABLE]

Clearly $(p_{t},q_{t})\equiv(0,0)$ is the solution. Therefore

[TABLE]

which makes the SMP useless in charactering the optimal control $u_{t}=0.$ ∎

Now, we discuss singular optimal stochastic controls defined as follows.

Definition 1.2**.**

An admissible control $\tilde{u}(\cdot)$ is singular on region $V$ if $V\subset U$ is of positive measure and for $a.e.\ t\in[0,T]$ and $v\in V$, we have for any $v\in V$,

[TABLE]

As we have seen in last example, the SMP is not very useful under singular control. Our goal is to derive further necessary condition for optimality. We shall call the original SMP as the first order SMP while the one we will derive as the second order one.

For second-order SMP of singular control problems, Bell [2], Gabasov [9], Kazemi-Dehkordi [12], Krener [13], Mizukami and Wu [19] devoted themselves to the deterministic case. Lu [17] interested in second order necessary conditions for stochastic evolution system. Tang [23] studied the singular optimal control problem for stochastic system with state equation

[TABLE]

and the cost functional

[TABLE]

By applying spike variation and vector-value measure theory, a second-order maximum principle is presented which involves the second-order adjoint process.

In this paper, we study the case when the state equation and the cost functional are in generalized mean-field form. The rest of this paper is organized as follows: In Section 2, we introduce the preliminaries about the generalized mean-field BSDEs. In Section 3, we set up the formulation of the singular optimal stochastic control problem and state the main result of the paper. Section 4 is devoted to the study of the impact of the control actions on the state and the cost functional by using Taylor’s expansion. In that section, we also present some estimations about the state. In Section 5, the method in Section 4 is reused for the expansion of the cost functional with respect to the control variable. Sections 6 is devoted to the proof of the second order stochastic maximum principle.

2. Preliminaries

In this section, for the convenience of the reader, we state some results of Buckdahn et al. [7] without proofs.

Let $\mathcal{P}_{2}(\mathbb{R}^{n})$ be the collection of all square integrable probability measures over $(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}))$, endowed with the 2-Wasserstein metric $W_{2}$, which is defined as

[TABLE]

for all $\mu^{\prime},\nu^{\prime}\in L^{2}(\mathcal{F}_{0};\mathbb{R}^{d})$ with $P_{\mu^{\prime}}=P_{\mu},\ P_{\nu^{\prime}}=P_{\nu}.$ Denote by $L^{2}(\mathcal{F};\mathbb{R}^{n})$ the collection of all $\mathbb{R}^{n}$-valued square integrable random variables. The following definition is taken from Cardaliaguet [8].

Definition 2.1**.**

A function $f:\mathcal{P}_{2}(\mathbb{R}^{n})\longrightarrow\mathbb{R}$ is said to be differentiable in $\mu\in\mathcal{P}_{2}(\mathbb{R}^{n})$ if, the function $\tilde{f}:L^{2}(\mathcal{F};\mathbb{R}^{n})\longrightarrow\mathbb{R}$ given by $\tilde{f}(\mathfrak{v})=f(P_{\mathfrak{v}})$ is differentiable (in Fréchet sense) at $\mathfrak{v}_{0}$, defined by $P_{\mathfrak{v}_{0}}=\mu$, i.e. there exists a linear continuous mapping $D\tilde{f}(\mathfrak{v}_{0}):L^{2}(\mathcal{F};\mathbb{R}^{n})\longrightarrow\mathbb{R},$ such that

[TABLE]

with $|\eta|_{L^{2}}\longrightarrow 0$ for $\eta\in L^{2}(\mathcal{F};\mathbb{R}^{n}).$

According to the Riesz representation theorem, there exists a unique random variable $\theta_{0}\in L^{2}(\mathcal{F};\mathbb{R}^{n})$ such that $D\tilde{f}(\mathfrak{v}_{0})(\eta)=(\theta_{0},\eta)_{L^{2}}=\mathbb{E}[\theta_{0}\eta]$, for all $\eta\in L^{2}(\mathcal{F};\mathbb{R}^{n}).$ In [8] it has been proved that there is a Borel function $h_{0}:\mathbb{R}^{n}\longrightarrow\mathbb{R}^{n}$ such that $\theta_{0}=h_{0}(\mathfrak{v}_{0})$ a.s. Then,

[TABLE]

We call $\partial_{\mu}f(P_{\mathfrak{v}_{0}},y):=h_{0}(y),\ y\in\mathbb{R}^{n}$, the derivative of $f:\mathcal{P}_{2}(\mathbb{R}^{n})\longrightarrow\mathbb{R}^{n}$ at $P_{\mathfrak{v}_{0}}.$ Note that $\partial_{\mu}f(P_{\mathfrak{v}_{0}},y)$ is $P_{\mathfrak{v}_{0}}(dy)$-$a.s.$ uniquely determined.

For mean-field type SDE and BSDE, we introduce the following notations. Let $(\Omega^{\prime},{\mathcal{F}}^{\prime},P^{\prime})$ be a copy of the probability space $(\Omega,{\mathcal{F}},P)$. For each random variable $\xi$ over $(\Omega,{\mathcal{F}},P)$ we denote by $\xi^{\prime}$ a copy of $\xi$ defined over $(\Omega^{\prime},{\mathcal{F}}^{\prime},P^{\prime})$. $\mathbb{E}^{\prime}[\cdot]=\int_{\Omega^{\prime}}(\cdot)dP^{\prime}$ acts only over the variables $\omega^{\prime}$.

Definition 2.2**.**

We say that $f\in C^{1,1}_{b}(\mathcal{P}_{2}(\mathbb{R}^{d}))$ (continuously differentiable over $\mathcal{P}_{2}(\mathbb{R}^{d})$ with Lipschitz-continuous bounded derivative), if for all $\mathfrak{v}\in L^{2}(\mathcal{F},\mathbb{R}^{d})$, there exists a $P_{\mathfrak{v}}$-modification of $\partial_{\mu}f(P_{\mathfrak{v}},\cdot)$, again denote by $\partial_{\mu}f(P_{\mathfrak{v}},\cdot)$, such that $\partial_{\mu}f:\mathcal{P}_{2}(\mathbb{R}^{d})\times\mathbb{R}^{d}\longrightarrow\mathbb{R}^{d}$ is bounded and Lipschitz continuous, i.e., there is a real constant $C$ such that

[TABLE]

we call this function $\partial_{\mu}f$ the derivative of $f$.

Given $f\in C^{1,1}_{b}(\mathcal{P}_{2}(\mathbb{R}^{d}))$, and $y\in\mathbb{R}^{d}$, the question of the differentiability of its components $(\partial_{\mu}f)_{j}(\cdot,y):\mathcal{P}_{2}(\mathbb{R}^{d})\rightarrow\mathbb{R},\ 1\leq j\leq d,$ raises. This can be discussed in the same way as the first order derivative $\partial_{\mu}f$ above. If $(\partial_{\mu}f)_{j}(\cdot,y):\mathcal{P}_{2}(\mathbb{R}^{d})\rightarrow\mathbb{R}$ belongs to $C^{1,1}_{b}(\mathcal{P}_{2}(\mathbb{R}^{d}))$, we have that its derivative $\partial_{\mu}((\partial_{\mu}f)_{j}(\cdot,y))(\cdot,\cdot):\mathcal{P}_{2}(\mathbb{R}^{d})\times\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ is a Lipschitz-continuous function. Then

[TABLE]

defines a function $\partial^{2}_{\mu}f:\mathcal{P}_{2}(\mathbb{R}^{d})\times\mathbb{R}^{d}\times\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}\otimes\mathbb{R}^{d}.$

Definition 2.3**.**

We say that $f\in C^{2,1}_{b}(\mathcal{P}_{2}(\mathbb{R}^{d}))$, if $f\in C^{1,1}_{b}(\mathcal{P}_{2}(\mathbb{R}^{d}))$ and

i) $(\partial_{\mu}f)_{j}(\cdot,y)\in C^{1,1}_{b}(\mathcal{P}_{2}(\mathbb{R}^{d}))$, for all $y\in\mathbb{R}^{d},\ 1\leq j\leq d$, and $\partial^{2}_{\mu}f:\mathcal{P}_{2}(\mathbb{R}^{d})\times\mathbb{R}^{d}\times\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}\otimes\mathbb{R}^{d}$ is bounded and Lipschitz-continuous;

ii) $(\partial_{\mu}f)(\mu,\cdot):\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ is differentiable for every $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d}),$ and its derivative $\partial_{y}\partial_{\mu}f:\mathcal{P}_{2}(\mathbb{R}^{d})\times\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}\otimes\mathbb{R}^{d}$ is bounded and Lipschitz-continuous.

Example 2.4**.**

For twice continuously differentiable functions $h:\mathbb{R}^{d}\rightarrow\mathbb{R}$ and $g:\mathbb{R}\rightarrow\mathbb{R}$ with bounded derivatives. Consider $f(P_{\mathfrak{v}}):=g(\mathbb{E}[h(\mathfrak{v})]),$ $\mathfrak{v}\in L^{2}(\mathcal{F};\mathbb{R}^{d}).$ Then, given any $\mathfrak{v}_{0}\in L^{2}(\mathcal{F};\mathbb{R}^{d}),$ $\tilde{f}(\mathfrak{v}):=f(P_{\mathfrak{v}})=g(\mathbb{E}[h(\mathfrak{v})])$ is Fréchet differentiable in $\mathfrak{v}_{0}$, and

[TABLE]

So, $D\tilde{f}(\mathfrak{v}_{0})(\eta)=\mathbb{E}[g^{\prime}(\mathbb{E}[h(\mathfrak{v}_{0})])h^{\prime}(\mathfrak{v}_{0})\eta],$ $\eta\in L^{2}(\mathcal{F};\mathbb{R}^{d}),i.e.,$

[TABLE]

Similarly, we see that

[TABLE]

and

[TABLE]

Let us now consider a complete probability space $(\Omega,\mathcal{F},P)$ on which we define a $d$-dimensional Brownian motion $B=(B^{1},\cdots,B^{d})=(B_{t})_{t\in[0,T]}$, where $T\geq 0$ denotes an arbitrarily fixed time horizon. We make the following assumptions: There is a sub-$\sigma$-field $\mathcal{F}_{0}\subset\mathcal{F}$ such that

i) the Brownian motion $B$ is independent of $\mathcal{F}_{0}$, and

ii) $\mathcal{F}_{0}$ is “rich enough”, i.e., $\mathcal{P}_{2}(\mathbb{R}^{d})=\{P_{\mathfrak{v}},\mathfrak{v}\in L^{2}(\mathcal{F}_{0};\mathbb{R}^{d})\}.$

By $\mathbb{F}=(\mathcal{F}_{t})_{t\in[0,T]}$ we denote the filtration generated by $B$, completed and augmented by $\mathcal{F}_{0}$.

Given deterministic Lipschitz functions $\sigma:\mathbb{R}^{d}\times\mathcal{P}_{2}(\mathbb{R}^{d})\longrightarrow\mathbb{R}^{d\times d}$ and $b:\mathbb{R}^{d}\times\mathcal{P}_{2}(\mathbb{R}^{d})\longrightarrow\mathbb{R}^{d},$ we consider for the initial state $(t,x)\in[0,T]\times\mathbb{R}^{d}$ and $\xi\in L^{2}(\mathcal{F}_{t};\mathbb{R}^{d})$ the stochastic differential equations (SDEs)

[TABLE]

and

[TABLE]

It is well-known that under the assumptions above both SDEs have unique solutions in $\mathcal{S}^{2}([t,T];\mathbb{R}^{d}),$ which is the space of $\mathbb{F}$-adapted continuous processes $Y=(Y_{s})_{s\in[t,T]}$ with $\mathbb{E}[\sup_{s\in[t,T]}|Y_{s}|^{2}]\leq\infty.$

Hypothesis 2.1**.**

*The couple of coefficients $(\sigma,b)$ belongs to $C^{1,1}_{b}(\mathbb{R}^{d}\times\mathcal{P}_{2}(\mathbb{R}^{d})\longrightarrow\mathbb{R}^{d\times d}\times\mathbb{R}^{d}),$ i.e., the components $\sigma_{i,j},b_{j},\ 1\leq i,j\leq d,$ satisfy the following conditions:

i) $\sigma_{i,j}(x,\cdot),b_{j}(x,\cdot)$ belong to $C^{1,1}_{b}(\mathcal{P}_{2}(\mathbb{R}^{d}))$, for all $x\in\mathbb{R}^{d}$

ii) $\sigma_{i,j}(\cdot,\mu),b_{j}(\cdot,\mu)$ belong to $C^{1}_{b}(\mathbb{R}^{d})$, for all $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$

iii) The derivatives $\partial_{x}\sigma_{i,j},\partial_{x}b_{j}:\mathbb{R}^{d}\times\mathcal{P}_{2}(\mathbb{R}^{d})\longrightarrow\mathbb{R}^{d}$, $\partial_{\mu}\sigma_{i,j},\partial_{\mu}b_{j}:\mathbb{R}^{d}\times\mathcal{P}_{2}(\mathbb{R}^{d})\times\mathbb{R}^{d}\longrightarrow\mathbb{R}^{d}$, are bounded and Lipschitz continuous.*

Hypothesis 2.2**.**

*The couple of coefficient $(\sigma,b)$ belongs to $C^{2,1}_{b}(\mathbb{R}^{d}\times\mathcal{P}_{2}(\mathbb{R}^{d})\longrightarrow\mathbb{R}^{d\times d}\times\mathbb{R}^{d}),$ i.e., $(\sigma,b)\in C^{1,1}_{b}(\mathbb{R}^{d}\times\mathcal{P}_{2}(\mathbb{R}^{d})\longrightarrow\mathbb{R}^{d\times d}\times\mathbb{R}^{d})$ and the components $\sigma_{i,j},b_{j},\ 1\leq i,j\leq d,$ satisfies the following conditions:

i) $\partial_{x_{k}}\sigma_{i,j}(\cdot,\cdot),\partial_{x_{k}}b_{j}(\cdot,\cdot)$ belong to $C^{1,1}_{b}(\mathbb{R}^{d}\times\mathcal{P}_{2}(\mathbb{R}^{d}))$, for all $1\leq k\leq d;$

ii) $\partial_{\mu}\sigma_{i,j}(\cdot,\cdot,\cdot),\partial_{\mu}b_{j}(\cdot,\cdot,\cdot)$ belong to $C^{1,1}_{b}(\mathbb{R}^{d}\times\mathcal{P}_{2}(\mathbb{R}^{d})\times\mathbb{R}^{d})$, for all $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$

iii) All the derivatives of $\sigma_{i,j},b_{j}$, up to order 2 are bounded and Lipschitz continuous.*

The following theorem is taken from [7]. It gives the Itô formula related to a probability measure.

Theorem 2.5**.**

Let $\Phi\in C^{2,1}_{b}(\mathbb{R}^{d}\times\mathcal{P}_{2}(\mathbb{R}^{d})).$ Then, under Hypothesis 2.2, for all $0\leq t\leq s\leq T,x\in\mathbb{R}^{d},\xi\in L^{2}(\mathcal{F}_{t};\mathbb{R}^{d})$ the Itô formula is satisfied as follow:

[TABLE]

For simplicity, we will make use of the following notations concerning matrices. We denote by $\mathbb{R}^{n\times d}$ the space of real matrices of $n\times d$-type, and by $\mathbb{R}^{n\times n}_{d}$ the linear space of the vectors of matrices $M=(M_{1},\cdots,M_{d})$, with $M_{i}\in\mathbb{R}^{n\times n}$, $1\leq i\leq d.$ Given any $\alpha,\beta\in\mathbb{R}^{n}$, $L,S\in\mathbb{R}^{n\times d}$, $\gamma\in\mathbb{R}^{d}$ and $M,N\in\mathbb{R}^{n\times n}_{d}$, we introduce the following notation: $\alpha\beta=\sum^{n}_{i=1}\alpha_{i}\beta_{i}\in\mathbb{R}$, $\alpha\times\beta=(\alpha_{i}\beta_{j})_{1\leq i,j\leq n}$; $LS=\sum^{d}_{i=1}L_{i}S_{i}\in\mathbb{R}$, where $L=(L_{1},\cdots,L_{d}),S=(S_{1},\cdots,S_{d})$; $ML=\sum^{d}_{i=1}M_{i}L_{i}\in\mathbb{R}^{n}$; $M\alpha\gamma=\sum^{d}_{i=1}(M_{i}\alpha)\gamma_{i}\in\mathbb{R}^{n}$; $MN=\sum^{d}_{i=1}M_{i}N_{i}\in\mathbb{R}^{n\times n}$;

For mean-field type SDE and BSDE, we have still to introduce some notations. Let $(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{P})$, $(\bar{\Omega},\bar{\mathcal{F}},\bar{P})$ be two copies of the probability space $(\Omega,{\mathcal{F}},P)$. For any random variable $\xi$ over $(\Omega,{\mathcal{F}},P)$, we denote by $\tilde{\xi}$ and $\bar{\xi}$ its copies on $\tilde{\Omega}$ and $\bar{\Omega}$, respectively, which means that they have the same law as $\xi$, but defined over $(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{P})$ and $(\bar{\Omega},\bar{\mathcal{F}},\bar{P})$. $\mathbb{\tilde{E}}[\cdot]=\int_{\tilde{\Omega}}(\cdot)d\tilde{P}$ and $\mathbb{\bar{E}}[\cdot]=\int_{\bar{\Omega}}(\cdot)d\bar{P}$ act only over the variables from $\tilde{\omega}$ and $\bar{\omega}$, respectively.

3. Formulation of the singular optimal stochastic control problem and the main result

In this section, we formulate our generalized mean-field optimal control problem and state the main result of this article. Let $(\Omega,{\mathcal{F}},P)$ be a probability space with filtration ${\mathcal{F}}_{t}$. Suppose that ${B}_{t}$ is a Brownian motion on $(\Omega,{\mathcal{F}},P)$, where ${\mathcal{F}}$ is the filtration generated by ${B}_{t}$, augmented by all $P$-null sets. Let $\mathcal{U}$ denote the admissible control set consisting of $\mathcal{F}_{t}$-adapted process $u_{t}$, take values in $U$, such that $\sup_{0\leq t\leq T}\mathbb{E}|u_{t}|^{8}<\infty$, where $U$ is a subset of $\mathbb{R}^{k}$. Let $b:[0,T]\times\mathbb{R}^{n}\times\ \mathcal{P}_{2}(\mathbb{R}^{n})\times U\longrightarrow\mathbb{R}^{n},\ \sigma:[0,T]\times\mathbb{R}^{n}\times\ \mathcal{P}_{2}(\mathbb{R}^{n})\longrightarrow\mathbb{R}^{n\times d},\ h:[0,T]\times\mathbb{R}^{n}\times\ \mathcal{P}_{2}(\mathbb{R}^{n})\times U\longrightarrow\mathbb{R}$, and $\Phi:\mathbb{R}^{n}\times\mathcal{P}_{2}(\mathbb{R}^{n})\longrightarrow\mathbb{R}.$

The state equation and the cost functional are defined by (1) and (2). Throughout this paper, we make the following assumptions on the coefficients:

Hypothesis 3.1**.**

*(1) The functions $b,\sigma,h,\Phi$ are differentiable with respect to $(x,\mu,v).$ $b,\sigma$ satisfy Lipschitz condition with respect to $(x,\mu,v).$

(2) The first-order derivatives with respect to $(x,\mu)$ of $b,\sigma$ are Lipschitz continuous and bounded.

(3) The first-order derivatives with respect to $(x,\mu)$ of $h,\Phi$ are Lipschitz continuous and bounded by $C(1+|x|+|v|)$.

(4) The second-order derivatives with respect to $(x,\mu)$ of $b,\sigma,h,\Phi$ are continuous and bounded. All the second-order derivatives are Borel measurable with respect to $(t,x,\mu,v).$*

Suppose that $u$ is an optimal control and $X^{u}$ is the associated trajectory. We are to find the necessary conditions satisfied by $u$. Firstly, we introduce the following abbreviations:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Similar shorthand notations for the second-order derivatives and those about $\sigma,h$ can also be introduced.

Consider the first order adjont process

[TABLE]

According to Theorem 3.1 [6], this BSDE admit a unique adapted solution. We also denote the solution as $(p^{u}_{t},q^{u}_{t}).$ Define the Hamiltonian as follows:

[TABLE]

The following first-order SMP is obtained as a special case of [5].

Theorem 3.1** (The First Order SMP).**

Let Hypothesis 3.1 hold. Suppose that $X^{u}_{t}$ is the associated trajectory of the optimal control $u$, and $(p,q)$ is the solution to the mean-field backward stochastic differential equation (MFBSDE) (12). Then, there is a subset $I_{0}\subset[0,T]$ which is of full measure such that $\forall t\in I_{0}$,

[TABLE]

As we pointed out in the introduction, the aim of this article is to derive another SMP when the Hamiltonian function above becomes singular, and hence, the SMP above is not suitable for characterizing of the optimal control $u_{t}$. To this end, we define the second-order adjoint process as follows:

[TABLE]

Remark 1**.**

By changing the terminal condition $p_{T}$, we can always eliminate the terminal cost when deducing the variational inequality. In fact, the terminal condition $P_{T}=0$ is due to the assumption that $\Phi\equiv 0$. Without this assumption, we only need to set

[TABLE]

Without loss of generality, we assume the terminal cost $\Phi\equiv 0$ in the following sections.

Finally, we present our main result in this article.

Theorem 3.2**.**

Assume that Hypothesis 3.1 hold. Let \big{(}X^{u}_{\cdot},u_{\cdot}\big{)} be an optimal pair and let $u_{\cdot}$ be singular on the control region $V$. Suppose that $(P,Q)$ is the unique adapted solution of equation (14). Then, there is a full measure subset $I_{0}\subset[0,T]$ such that at each $t\in I_{0}$, \big{(}X^{u}_{\cdot},u_{\cdot}\big{)} satisfies, not only the first-order stochastic maximum principle, but also the following inequality

[TABLE]

4. Quantitative analysis of the impact of control actions on the state

In this section, we expand the state process according to different orders of the perturbation parameter $d(u,v)$, a distance between the optimal control $u$ and its perturbation $v$.

Lemma 4.1**.**

Under Hypothesis 3.1 on the coefficients, we have,

[TABLE]

Proof.

By the state equation (1), for $\tau\in[0,T]$ we have,

[TABLE]

From Gronwall’s inequality, we then have the desired result. ∎

For $v_{i}\in\mathcal{U}$, $i=1,2$, we define

[TABLE]

and $d(v_{1},v_{2})=|I(v_{1},v_{2})|$ is the Lebesgue measure of $I(v_{1},v_{2})$. Then, $(\mathcal{U},d)$ is a metric space.

Given the optimal pair $(X^{u}_{\cdot},u_{\cdot})$, we now proceed to the perturbation $X^{v}$ of $X^{u}$.

Let

[TABLE]

and

[TABLE]

Denote

[TABLE]

The following lemmas give the estimation of their orders according to parameter $d(v,u)$.

Lemma 4.2**.**

Assume that Hypothesis 3.1 holds. Then, there exists a $K>0$, such that for any $v(\cdot),u(\cdot)\in\mathcal{U}$, we have

[TABLE]

Proof.

For any $\tau\in[0,T]$, denote

[TABLE]

By Hypothesis 3.1 and the Burkholder-Davis-Gundy inequality, we have

[TABLE]

and

[TABLE]

The application of Grownwall’s inequality allows to obtain that

[TABLE]

Notice that the first-order derivative $b_{x}$ is bounded. Then, (26) implies the following estimate

[TABLE]

According to assumption about $v$ and $u$, then,

[TABLE]

In fact, by Minkowski’s inequality, we have

[TABLE]

∎

The following lemma gives the order of ${X^{v*}_{t}}$.

Lemma 4.3**.**

Assume Hypothesis 3.1 holds. For $v(\cdot)\in\mathcal{U}$ and Borel subset $I_{\rho}\subset[0,T]$ with Lebesgue measure $|I_{\rho}|$, define

[TABLE]

Then we have

[TABLE]

when $|I_{\rho}|\rightarrow 0$.

Proof.

We introduce the following notations first

[TABLE]

[TABLE]

[TABLE]

[TABLE]

where $X^{v,12}_{\cdot}:=X^{v,1}_{\cdot}+X^{v,2}_{\cdot}.$ Similarly notations can be introduced with $b$ replaced by $\sigma.$

We now proceed to estimating $X^{*}_{.}(\hat{v})$ defined by (20). By (1), (18) and (19), we have

[TABLE]

where

[TABLE]

[TABLE]

We can represent $\alpha(t)$ as follows.

[TABLE]

Denote

[TABLE]

It is easy to show that

[TABLE]

Simularly, by setting

[TABLE]

we have

[TABLE]

According to Hypothesis 3.1, we have

[TABLE]

Note that

[TABLE]

By Burkholder-Davis-Gundy inequality, for $\tau\in[0,T]$, we obtain the following estimation

[TABLE]

According to Gronwall’s inequality, we have

[TABLE]

About $A(s;{\hat{v}})$ we have

[TABLE]

Note that

[TABLE]

similar estimates hold with $b$ replaced by $b_{x}$ and $b_{\mu}$. Since $X^{{\hat{v}},12}_{t}\rightarrow 0$ as $|I_{\rho}|\rightarrow 0$, so we also have

[TABLE]

replace $b_{xx}$ by $b_{\mu\mu}$, $b_{\mu y}$ and $b_{x\mu}$, we can get the similar result when $|I_{\rho}|\rightarrow 0$.

According to estimation of $X^{{\hat{v}},1}_{\cdot},\ X^{{\hat{v}},2}_{\cdot}$ in Lemma 4.2, we obtain

[TABLE]

Similarly

[TABLE]

Finally, by (37) we have the desire result. ∎

5. Expansion of the cost functional with respect to control variable

In this section, we use the method of Lemma 4.3 again to study the expansion of the cost functional according to different order of the purtubation.

Lemma 5.1**.**

Assume that Hypothesis 3.1 holds. Define

[TABLE]

Recall that $\hat{v}$ is defined by (30) by $I_{\rho}$. Then, when $|I_{\rho}|\rightarrow 0$, we have

[TABLE]

Proof.

Denote

[TABLE]

By (20) and Lemma 4.3, we have

[TABLE]

where

[TABLE]

Then,

[TABLE]

and hence,

[TABLE]

Using the same method in Lemma 4.3, we complete the proof. ∎

Now, we proceed to deriving the expansion of the perturbed cost function.

[TABLE]

Recalling that $p_{t}$ is given by (12) and applying Itô’s formula to $p_{t}X^{v,12}_{t}$, we obtain

[TABLE]

Hence,

[TABLE]

Now we apply the range theorem for vector-valued measures due to [18], to deduce the variational inequality.

Recall that ${\hat{v}}_{t}$ is defined by (30). According to Lemma 4.1 [18], for any $0<\rho<1$, we can choose a suitable $I_{\rho}\subset[0,T]$ such that $|I_{\rho}|=\rho T,$

[TABLE]

[TABLE]

and

[TABLE]

Lemma 5.2**.**

For the $I_{\rho}$ above, $t\in[0,T]$, we also have

[TABLE]

The proof of the above lemma is essentially the same as Lemma 4.1 [18]. For the convenience of readers, we present the proof here.

Proof.

Let $\phi_{i}(\cdot)\in L^{2}\left(\Omega;L^{2}(0,T;\mathbb{R}^{l_{i}})\right),$ $l_{i}\geq 1$, $i=1,\cdots,k.$ Suppose

[TABLE]

. Given $0<\rho<1$ and set $\delta=\rho^{5}$, then there exists a $n>0$, we can find a process $\phi^{\rho}_{i}(\cdot)$ in the form of

[TABLE]

with $0=t_{0}<t_{1}<\cdots<t_{n+1}=T,$ $\max|t_{i+1}-t_{i}|<\delta$, $\xi^{j}_{i}$ being $\mathcal{F}_{t_{j}}$-measurable, s.t.

[TABLE]

Note that we can always choose the partition $\{t_{j}\}_{0\leq j\leq n+1}$ independent of $i=1,\cdots,k.$ Now letting

[TABLE]

It’s easy to see $|G|=\rho T$. Thus (53), (54), and (55) are proved by taking $\phi_{i}$ suitably. For any $s\in[0,T]$, we can always find a $m\geq 0$, s.t. $s\in[t_{m},t_{m+1}).$ Then we have

[TABLE]

where $I_{G}(t)$ is the indicator function of $G$. Thus, for each $1\leq i\leq k,$

[TABLE]

Set $\phi_{1}(t)=\bigtriangleup b(t,v)$, then letting $\rho\rightarrow 0$, we finish the proof. ∎

Lemma 5.3**.**

For any $\rho\in[0,1]$, there exists a subset $I_{\rho}$ of $[0,T]$, such that

[TABLE]

where $\hat{v}$ is defined by (30) with $I_{\rho}$ given here.

Proof.

By (18) and lemma 5.2, we have

[TABLE]

Thus, we can get

[TABLE]

Notice that

[TABLE]

By Gronwall’s inequality, we get (61). ∎

Further more, inspired by [24], we also have the following lemma holds.

Lemma 5.4**.**

For any $\rho\in[0,1]$, there exists a set $I_{\rho}\in[0,T]$ and a matrix value process $\Phi(t)$, s.t. $X^{\hat{v},1}_{t}$ is represented by the following

[TABLE]

where

[TABLE]

Proof.

Let $\Phi(t)$ be the unique solution of the following matrix value SDE:

[TABLE]

and set

[TABLE]

Applying Itô’s formula to $[\Psi_{t}\Phi_{t}]$, we can easily get $d[\Psi_{t}\Phi_{t}]=0$, which means $\Psi_{t}\Phi_{t}\equiv I$, i.e. $\Psi_{t}=\Phi_{t}^{-1}.$ Applying Itô’s formula to $d\left[\Psi_{t}X^{\hat{v},1}_{t}\right]$, by Lemma 5.3, we then get our desire result. ∎

Now we continue to derive the expansion. Applying Itô’s formula to $Y_{t}:=X^{\hat{v},1}_{t}{X}^{*\hat{v},1}_{t}$, we have

[TABLE]

Applying Itô’s formula to $P_{t}Y_{t}$, according to (61), we obtain

[TABLE]

Putting (70) into (52), we conclude

[TABLE]

From Lemma 5.1, (61) and (71), we obtain

[TABLE]

Finally, according to (55), we have

[TABLE]

6. The proofs of the stochastic maximum principle.

Although the first-order SMP has been obtained by [5], we give a proof here for completeness. In fact, after the preparation of the previous sections which will also be needed in the proof of the second-order SMP, this proof does not take too much extra effort.

Proof of first-order SMP: Since \big{(}X^{u}_{\cdot},u_{\cdot}\big{)} is an optimal pair of our system, it follows from (73) that

[TABLE]

for any $\rho\in[0,T]$, $\forall v(\cdot)\in\mathcal{U}$. Setting $\rho\rightarrow 0+$, we obtain

[TABLE]

Then we can deduce that, for any fixed $v\in\mathcal{U}$, there exists a null subset $S^{v}\subset[0,T]\times\Omega$, such that for each $(t,\omega)\in\left(S^{v}\right)^{c}$,

[TABLE]

Otherwise, suppose that

[TABLE]

has positive measure in $[0,T]\times\Omega$, for a $v^{*}\in\mathcal{U}.$ Let

[TABLE]

Then,

[TABLE]

This contradicts from (75).

Select a countable dense subset $\{v_{s}^{(i)}\}^{\infty}_{i=1}\subset U,$ set

[TABLE]

Then, $S_{0}$ is a null subset of $[0,r]\times\Omega$, and for $(t,\omega)\in S:=\left(S_{0}\right)^{c}$, we get

[TABLE]

By Fubini’s theorem, it is easy to see that there exists a null subset $T_{0}$ of $[0,T]$, such that $\forall t\in T_{0}^{c},$ (78) holds $a.s.$.

Finally, from the continuity of the function and the denseness of $\{v^{(i)}\}^{\infty}_{i=1}$, we have for $t\in\left(T_{0}\right)^{c}$,

[TABLE]

∎

Now, we proceed to presenting the proof of the second-order stochastic maximum principle for singular generalized mean-field control problem.

Proof of Theorem 3.2. The optimality and the singularity imply that

[TABLE]

According to (73), we have

[TABLE]

where

[TABLE]

As in [11][23], denote by $\{t_{i}\}^{\infty}_{i=1}$ the collection of all rational numbers in $[0,T],$ and $\{v_{k}\}^{\infty}_{k=1}$ a dense subset of $V$. Because of the fact that $\mathcal{F}_{t}$ is countability generated for $t\in[0,T],$ we can assume $\{A_{i,j}\}^{\infty}_{j=1}$ generates $\mathcal{F}_{t_{i}},\ i=1,2,\cdots.$ For any $\tau\in[t_{i},T)$ and $\theta\in(0,T-\tau),$ write $E^{i}_{\theta}=[\tau,\tau+\theta),$ and define

[TABLE]

Let $X^{1k}_{ij}$ be the solution to the equation (18) with respect to $v^{k}_{i,j}(\cdot)$. Notice that we can always choose suitable $I_{\theta}$, such that $I_{\theta}\cap E^{i}_{\theta}=E^{i}_{\theta}.$ So Lemma 5.4 holds for $X^{1k}_{ij}$.

By Lemma 4.1 [25], lemma 4.2 and Lebesgue differential theorem, there is a null subset $T^{k}_{ij}\subset[0,T]$ such that for $\tau\in\left(T^{k}_{ij}\right)^{c}$, we have

[TABLE]

Set

[TABLE]

Then, $T_{0}$ is a null subset of $[0,T]$. For $s\in[0,T]\setminus T_{0}$ and $i$, we deduce that

[TABLE]

which means

[TABLE]

By virtue of the continuity of the function and the denseness of $\{v_{k}\}^{\infty}_{k=1}$, we finish the proof.∎

Remark 2**.**

We now come back to Example 1.1. It is not hard to check that the second order adjoint process $(P_{t},Q_{t})\equiv(1,0).$ Then

[TABLE]

So we can say $u_{t}\equiv 0$ is the only candidate for optimal controls.