Mean-field optimal control problem of SDDEs driven by fractional Brownian motion

TL;DR

This paper develops a framework for solving mean-field optimal control problems involving SDDEs driven by fractional Brownian motion with Hurst parameter greater than 0.5, using fractional White noise calculus and measure differentiation.

Contribution

It introduces necessary and sufficient stochastic maximum principles for such control problems, overcoming the limitations of classical methods.

Findings

Derived stochastic maximum principles for fractional Brownian motion-driven SDDEs.

Solved an optimal consumption problem with delay.

Addressed a linear-quadratic control problem with delay.

Abstract

We consider a mean-field optimal control problem for stochastic differential equations with delay driven by fractional Brownian motion with Hurst parameter greater than one half. Stochastic optimal control problems driven by fractional Brownian motion can not be studied using classical methods, because the fractional Brownian motion is neither a Markov process nor a semi-martingale. However, using the fractional White noise calculus combined with some special tools related to the differentiation for functions of measures, we establish and prove necessary and sufficient stochastic maximum principles. To illustrate our study, we consider two applications: we solve a problem of optimal consumption from a cash flow with delay and a linear-quadratique (LQ) problem with delay.

Equations

$$\begin{array}[c]{lll}dX(t)&=&b(t,X(t),X(t-\delta),M(t),M(t-\delta),u(t))dt+\sigma(t,M(t),M(t-\delta))dB^{H}_{t},\text{ }t\in\left[0,T\right],\text{\ }\\ X(t)&=&x_{0}(t);\text{ \ }t\in\left[-\delta,0\right],\end{array}$$

$$M(t):={\mathbb{P}}_{X(t)},M(t-\delta ):={\mathbb{P}}_{X(t-\delta )}$$

$$b:\Omega \times [0,T]\times \mathbb{R}\times \mathbb{R}\times {\mathcal{P}}_2(\mathbb{R})\times {\mathcal{P}}_2(\mathbb{R})\times \mathcal{U}\to \mathbb{R},$$

$$\sigma :\left[0,T\right]\times {\mathcal{P}}_2(\mathbb{R})\times {\mathcal{P}}_2(\mathbb{R})\to \mathbb{R},$$

$$\begin{array}[c]{lll}J(u)&=&\mathbb{E[}g(X(T),M(T))+{\textstyle\int_{0}^{T}}f(t,X(t),X(t-\delta),M(t),M(t-\delta),u(t))dt]\end{array},$$

$$\begin{array}[c]{lll}\mathbb{E[}|g(X(T),M(T))|+{\textstyle\int_{0}^{T}}|f(t,X(t),X(t-\delta),M(t),M(t-\delta),u(t))|dt]<+\infty.\end{array},$$

$$J(u^{\ast })=\underset{u\in {\mathcal{A}}_{\mathbb{F}}}{\sup }J(u).$$

$${\phi }_H(t,s)=H(2H-1)|t-s{|}^{2H-2},s,t\in [0,T].$$

$$\Vert f{\Vert }_H^2:={\int }_0^T{\int }_0^{T}f(s)f(t){\phi }_H(t,s)dsdt<+\infty .$$

$${\int }_{\Omega }{e}^{i<\omega ,f>}d\mathbb{P}(\omega )=e^{-1/2\Vert f{\Vert }_H^2}\forall f\in \mathcal{S}([0,T])$$

$$\mathbb{E}[<.,f>]=0\text{and}\mathbb{E}[<.,f{>}^2]=\Vert f{\Vert }_H^2$$

$$B_t^H:=<\omega ,{\chi }_{[0,t]}(.)>$$

$$\mathbb{E}[B_t^H{B}_s^H]=\frac{1}{2}(t^{2H}+s^{2H}-|t-s{|}^{2H}),s,t\in [0,T]$$

$$\mathbb{E}[{\int }_0^T{f}_{s}d{B}_s^H{\int }_0^T{g}_{s}d{B}_s^H]={\int }_0^T{\int }_0^{T}f(s)g(t){\phi }_H(t,s)dtds$$

$$F=g(B^H({\psi }_1),\cdots ,B^H({\psi }_n)),$$

$$D_s^{H}F:=\sum _{i=1}^n\frac{\partial g}{\partial x_i}(B^H({\psi }_1),\cdots ,B^H({\psi }_n)){\psi }_i(s),0\le s\le T.$$

$${\mathbb{D}}_t^{H}F:={\int }_0^T{D}_r^{H}F{\phi }_H(r,t)dr,$$

$$\Vert G{\Vert }_{{\mathcal{L}}_H^{1,2}}^2:=\mathbb{E}[{\int }_0^T{\int }_0^{T}G(s)G(t){\phi }_H(t,s)dtds+({\int }_0^T{\mathbb{D}}_s^{H}G(s)ds{)}^2]<\infty .$$

$$G(t,\omega )={\sum }_{i=1}^N{G}_i(\omega ){\chi }_{[t_i,t_{i+1}[}(t),$$

$${\int }_0^{T}G(t,\omega )d{B}^H(t):={\sum }_{i=1}^N{G}_i(\omega )♢(B^H(t_{i+1})-B^H(t_i)),$$

$$\mathbb{E}[({\int }_0^{T}G(t)d{B}^H(t){)}^2]=\mathbb{E}[{\int }_0^T{\int }_0^{T}G(s)G(t){\phi }_H(t,s)dtds+({\int }_0^T{\mathbb{D}}_s^{H}G(s)ds{)}^2],$$

$$\mathbb{E}[{\int }_0^T{G}_1(t)d{B}^H(t){\int }_0^T{G}_2(t)d{B}^H(t)]=\mathbb{E}[{\int }_0^T{G}_1(s)G_2(t){\phi }_H(s,t)dsdt+{\int }_0^T{\int }_0^T{\mathbb{D}}_s^H{G}_1(s){\mathbb{D}}_t^H{G}_2(t)dsdt]$$

$$\mathbb{E}[{\int }_0^{T}G(t)d{B}^H(t)]=0;\text{ for all}G\in {\mathcal{L}}_H^{1,2}([0,T]).$$

$$dX(t)=F_1(t)dt+G_1(t)d{B}^H(t),X(0)=x\in \mathbb{R},$$

$$dY(t)=F_2(t)dt+G_2(t)d{B}^H(t),Y(0)=y\in \mathbb{R},$$

$$\mathbb{E}[X(T)Y(T)]$$

$$+\mathbb{E}[{\int }_0^T{\int }_0^T{G}_1(t)G_2(s){\phi }_H(t,s)dsdt]+\mathbb{E}[{\int }_0^T{\int }_0^T{\mathbb{D}}_t^H{G}_1(t){\mathbb{D}}_s^H{G}_2(s)dsdt],$$

$$\mathbb{E}[X(T)Y(T)]=xy+\mathbb{E}[{\int }_0^{T}X(s)dY(s)]+\mathbb{E}[{\int }_0^{T}Y(s)dX(s)]+\mathbb{E}[{\int }_0^T{\int }_0^T{G}_1(t)G_2(s){\phi }_H(t,s)dsdt].$$

$${\mathcal{P}}_p(\mathbb{R})=\{m\in \mathcal{P}(\mathbb{R}):{\int }_{\mathbb{R}}|x{|}^{p}m(dx)<+\infty \}.$$

$$\displaystyle W_{p}(m,m^{\prime})=\inf\bigg{\{}\left(\int_{\mathbb{R}^{2}}|x-y|^{p}\rho(dx,dy)\right)^{\frac{1}{p}},$$

$$\displaystyle\rho(\cdot\times\mathbb{R})=m\ \hbox{and}\ \rho(\mathbb{R}\times\cdot)=m^{\prime}\bigg{\}}.$$

Full text

Keywords Mean-Field, Stochastic Delayed Differential Equations, Fractional Brownian Motion, Stochastic Maximum Principles.

**Mathematics Subject Classification  **60G22. 60H07. 60H04. 93E20

1 Introduction

The interest for stochastic delayed differential equations is constantly increasing. They are frequently used to model the evolution of systems with past-dependence nature. Such systems usually appear in biology, engineering and mathematical finance.

There is a rich litterature on stochastic optimal problems with delay. A lot of authors studied both the case where the stochastic systems are driven by a classical Brownian motion as well as where there is jumps, see,e.g., [1, 2, 3, 4].

Stochastic control problems driven by fractional Brownian motion (fBm) were also studied by many authors, see,e.g., [5, 6, 7, 8]. However, compared with the papers on stochastic control problems driven by the classical Brownian motion, few has been done because classical methods to solve control problems can not be used direclty, since the fractional Brownian motion is not a semi-martingale and not a Markov process.

Mean-field problems have also attracted wide attention recently, due to their several applications in physics, economics, finance and stochastic games. Mean-field games were first studied by Lasry and Lions [9]. Buckdhan, Li and Peng [10] studied a special mean-field games and introduced the so-called mean-field backward stochastic differential equations. Later, Carmona and Delarue [11] studied a class of mean-field forward-backward stochastic differential equations and gave many applications.

In this paper, all the previous fields are combined to study the optimal control problem of mean-field stochastic differential equations driven by fractional Brownian motion. The dynamic of the controlled state process depends on the state, the control, their laws but also on their values at previous time instants.

The dynamics of this work are close to those in the paper of Qiuxi Wang, Feng Chen and Fushan Huang [8]. In [8], the adjoint equation is an anticipated backward stochastic diferential equation (ABSDE) driven by both a fBm and a standard Brownian motion and the integral with respect to the fBm, is defined in the Stratonovich sense.

This work is also inspired by the recent paper of Buckdhan Rainer and Shuai Jing [6], if the system has a past-dependence feature. In [6] the dynamic of the adjoint process is driven by a standard Brownian motion, here the anticipated BSDE is driven by a fBM.

The approach used here is an extension of the work of Biagini, Hu, Øksendal and Sulem [5] when the dynamic of the system is of delayed mean-field type.

In our paper, we establish and proove necessary and sufficient maximum principles and we illustrate our study by solving two optimal control problems : a mean-field optimal consumption problem from a cash flow with delay and a linear-quadratique (LQ) problem with delay.

We present now more specifically the general problem we consider:

2 Statement of the problem

Let $B^{H}$ be a fractional Brownian motion on a filtred probability space $(\Omega,\mathcal{F},\mathbb{F}=(\mathcal{F}_{t})_{t\geq 0},\mathbb{P})$.

We consider a mean-field controlled stochastic delay equation of the form:

[TABLE]

where

[TABLE]

and $T>0$, $\delta>0$ are given constants. Here

[TABLE]

[TABLE]

are given functions such that, for all $t\in[0,T]$, $b(.,t,x,\bar{x},m,\bar{m},u)$ is supposed to be $\mathcal{F}_{t}$- measurable for all $x,\bar{x}\in\mathbb{R}$, $u\in\mathcal{U}$, $m,\bar{m}\in\mathcal{P}_{2}(\mathbb{R})$. The function $\sigma$ is assumed to be deterministic such that its integral with respect to the fBm will be a Wiener type integral. $\mathcal{P}_{2}(\mathbb{R})$ denotes the space of all probability measures $m$ on $(\mathbb{R},\mathcal{B}(\mathbb{R}))$, such that $\int_{\mathbb{R}}|x|^{2}m(dx)<\infty$.

The function $x_{0}$ is assumed to be continuous and deterministic. The set $\mathcal{U}\subset\mathbb{R}$ consists of the admissible control values. The information available to the controller is given by the filtration $\mathbb{F}$ generated by the fBm $B^{H}$. The set of admissible controls denoted by $\mathcal{A}_{\mathbb{F}}$ are the strategies available to the controller, required to be $\mathcal{U}$-valued and $\mathbb{F}$-adapted processes. Through the paper, we assume that $X$ exists and belongs to $L^{2}(\Omega\times[0,T])$.

The performance functional is assumed to have the form

[TABLE]

where $f:\Omega\times\left[0,T\right]\times\mathbb{R}\times\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R})\times\mathcal{P}_{2}(\mathbb{R})\times\mathcal{U}\rightarrow\mathbb{R}$ and $g:\Omega\times\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R})\rightarrow\mathbb{R}$ are given functions, such that for all $t\in[0,T]$, $f(.,t,x,\bar{x},m,\bar{m},u)$ is assumed to be $\mathcal{F}_{t}$-measurable for all $x,\bar{x}\in\mathbb{R}$, $u\in\mathcal{U}$, $m,\bar{m}\in\mathcal{P}_{2}(\mathbb{R})$. The function $g(.,x,m)$ is assumed to be $\mathcal{F}_{T}$-measurable for all $x\in\mathbb{R}$, $m\in\mathcal{P}_{2}(\mathbb{R})$.

We also assume the following integrability condition

[TABLE]

The functions $\sigma$, $b$, $f$ and $g$ are assumed to be continuously differentiable with respect to $x,\bar{x},u$ with bounded derivatives and admit Fréchet bounded derivatives with respect to $m,\bar{m}$.

The problem we consider in this paper is the following :

Problem: Find a control $u^{*}\in\mathcal{A}_{\mathbb{F}}$ such that

[TABLE]

any control $u^{*}\in\mathcal{A}_{\mathbb{F}}$ satisfying (1) is called an optimal control.

3 Generalities

In this section we give some preliminaries concerning fractional Brownian motion based on fractional White noise calculus and some generalities on differentiability with respect to the measures. For a general introduction to fractional White noise theory the reader may consult the books [12, 13].

3.1 Fractional Brownian motion

Let $T>0$ be a finite time horizon and $\Omega$ be the space $\mathcal{S}^{{\prime}}([0,T])$ of tempered distributions on $[0,T]$, which is the dual of the Schwartz space $\mathcal{S}([0,T])$ of rapidly decreasing smooth functions on $[0,T]$.

For $1/2<H<1$. We put

[TABLE]

If $\omega\in\mathcal{S}^{{\prime}}([0,T])$ and $f\in\mathcal{S}([0,T])$ , we let $<\omega,f>=\omega(f)$ denote the action of $\omega$ applied to $f$, it can be extended to $f:[0,T]\rightarrow\mathbb{R}$ such that

[TABLE]

The space of all such functions $f$ is denoted by $\mathbb{L}_{H}^{2}([0,T])$. The map $f\mapsto\exp(-1/2\|f\|_{{H}}^{2})$ with $f\in\mathcal{S}([0,T])$, is positive definite on $\mathcal{S}([0,T])$, then by Bochner-Minlos theorem there exists a probability measure $\mathbb{P}^{H}=\mathbb{P}$ on the Borel subsets $\mathcal{B}(\Omega)$ such that

[TABLE]

It follows from (2) that

[TABLE]

where $\mathbb{E}$ denotes the expectation under the probability measure $\mathbb{P}$. Hence, if we put

[TABLE]

then using (3), $B^{H}$ is a fractional Brownian motion with Hurst parameter $H$, that is a centred Gaussian process with covariance function

[TABLE]

From now on, we endow $\Omega$ with the natural filtration $\mathbb{F}:=\{\mathcal{F}_{t}\}_{t\in[0,T]}$ generated by $B^{H}$.

Lemma 3.1**.**

If $f,g$ belong to $\mathbb{L}_{H}^{2}([0,T])$, then the Wiener integrals $\int_{0}^{T}f_{s}dB^{H}_{s}$ and $\int_{0}^{T}g_{s}dB^{H}_{s}$ are well defined as zero mean Gaussian random variables with variances $\|f\|_{{H}}^{2}$ and $\|g\|_{{H}}^{2}$ respectively and

[TABLE]

We denote by $\tilde{\mathcal{S}}$ the set of all polynomial functions of $B^{H}(\psi_{j})=\int^{T}_{0}\psi_{j}(t)dB^{H}(t)$. For an element $F\in\tilde{\mathcal{S}}$, having the form

[TABLE]

where $g$ is a polynomial function of $n$ variables, we define its Malliavin derivative $D_{s}^{H}F$ by

[TABLE]

We also introduce another derivative

[TABLE]

Let $\mathcal{L}_{H}^{1,2}([0,T])$ be the set of processes $G:[0,T]\times\Omega\rightarrow\mathbb{R}$ such that $\mathbb{D}_{s}^{H}G(s)$ exists for all $s\in[0,T]$ and

[TABLE]

We let $\int_{0}^{T}G(s)dB^{H}(s)$ denote the fractional Wick-Itô-Skorohod (fWIS) integral of the process $G$ with respect to $B^{H}$. We recall its construction: if $G$ belongs to the family $\mathbb{S}$ of step functions of the form

[TABLE]

where $0\leq t_{1}<t_{2}<...<t_{N+1}\leq T$, then the fWIS integral is defined naturally as follows

[TABLE]

where $\diamondsuit$ denotes the Wick product, see [12] for its definition. For $G\in\mathbb{S}\cap\mathcal{L}_{H}^{1,2}([0,T])$, we have the isometry

[TABLE]

Using this we can extend the integral ${\textstyle\int_{0}^{T}}G(t)dB^{H}(t)$ to $\mathcal{L}_{H}^{1,2}([0,T])$. Note that if $G_{1},G_{2}\in\mathcal{L}_{H}^{1,2}([0,T])$, we have by polarization

[TABLE]

An important property of this integral is that

[TABLE]

We will need the following integration by parts formula.

Proposition 3.1** (Integration by parts).**

Let $X$ and $Y$ be two processes of the form

[TABLE]

and

[TABLE]

where $F_{1}:[0,T]\times\Omega\rightarrow\mathbb{R}$, $F_{2}:[0,T]\times\Omega\rightarrow\mathbb{R}$, $G_{1}:[0,T]\times\Omega\rightarrow\mathbb{R}$ and $G_{2}:[0,T]\times\Omega\rightarrow\mathbb{R}$ are given processes such that $G_{1}$, $G_{2}\in\mathcal{L}_{H}^{1,2}([0,T])$.

Then, for $T>0$,

[TABLE]

provided that the first two integrals exist. 2. 2.

In particular, if $G_{1}$ or $G_{2}$ is deterministic, then

[TABLE]

3.2 Differentiability of Functions of Measures

Let $\mathcal{P}(\mathbb{R})$ be the space of all probability measures on $(\mathbb{R},\mathcal{B}(\mathbb{R}))$. We denote by $\mathcal{P}_{p}(\mathbb{R})$ the subspace of $\mathcal{P}(\mathbb{R})$ of order $p$, which means

[TABLE]

• $\bullet$

The Wasserstein metric :

On $\mathcal{P}_{p}(\mathbb{R})$, the Wasserstein metric of order $p$ is defined by

[TABLE]

Notice that if $\xi$ and $\eta$ are two real $p$-integrable random variables with laws $\mathbb{P}_{\xi}$ and $\mathbb{P}_{\eta}$, then we have $W_{p}(\mathbb{P}_{\xi},\mathbb{P}_{\eta})\leq\left(\mathbb{E}[{|\xi-\eta|^{p}}]\right)^{\frac{1}{p}}$ since we can choose a special $\rho=\mathbb{P}_{(\xi,\eta)}$ in the above definition.

• $\bullet$

Diffentiability of functions of measures :

The notion of differentiability for functions of measures that we will use in the paper, is the one introduced by Lions in his course at Collège de France [14] and summarized by Cardaliaguet [15]. We also refer to Carmona and Delarue [11].

It’s based on the lifting of fuctions $m\in\mathcal{P}_{2}(\mathbb{R})\mapsto\sigma(m)$ into functions $\tilde{\xi}\in L^{2}(\tilde{\Omega};\mathbb{R})\mapsto\tilde{\sigma}(\tilde{\xi})$, over some probability space $(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})$, by setting $\tilde{\sigma}(\tilde{\xi}):=\sigma(\tilde{\mathbb{P}}_{\tilde{\xi}})$.

Definition 3.2**.**

A function $\sigma$ is said to be differentiable at $m_{0}\in\mathcal{P}_{2}(\mathbb{R})$, if there exists a random variable $\tilde{\xi}_{0}\in L^{2}(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})$ over some probability space $(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})$ with $\tilde{\mathbb{P}}_{\tilde{\xi}_{0}}=m_{0}$ such that $\tilde{\sigma}:L^{2}(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})\to\mathbb{R}$ is Fréchet differentiable at $\tilde{\xi}_{0}$.

We suppose for simplicity that $\tilde{\sigma}:L^{2}(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})\to\mathbb{R}$ is Fréchet differentiable. We denote its Fréchet derivative at $\tilde{\xi}_{0}$ by $D\tilde{\sigma}(\tilde{\xi}_{0})$. Recall that $D\tilde{\sigma}(\tilde{\xi}_{0}):L^{2}(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})\to\mathbb{R}$ is a continuous linear mapping; i.e. $D\tilde{\sigma}(\tilde{\xi}_{0})\in L(L^{2}(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}}),\mathbb{R})$. With the identification that $L(L^{2}(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}}),\mathbb{R})\equiv L^{2}(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})$ given by Riesz representation theorem, $D\tilde{\sigma}(\tilde{\xi}_{0})$ is viewed as an element of $L^{2}(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})$, hence we can write

[TABLE]

where $\tilde{\xi}$ is a random variable with law $m$. Moreover, according to Cardaliaguet [15], there exists a Borel function $h_{m_{0}}:\mathbb{R}\to\mathbb{R}$, such that $D\tilde{\sigma}(\tilde{\xi}_{0})=h_{m_{0}}(\tilde{\xi}_{0})$, $\tilde{\mathbb{P}}$-a.s. We define the derivative of $\sigma$ with respect to the measure at $m_{0}$ by putting $\partial_{m}\sigma(m_{0})(x):=h_{m_{0}}(x)$. Notice that $\partial_{m}\sigma(m_{0})(x)$ is defined $m_{0}(dx)$-a.e. uniquely. Therefore, the following differentiation formula is invariant by modification of the space $\tilde{\Omega}$ where the random variables $\tilde{\xi}_{0}$ and $\tilde{\xi}$ are defined, i.e.

[TABLE]

whenever $\tilde{\xi}$ and $\tilde{\xi}_{0}$ are random variables with laws $m$ and $m_{0}$ respectively.

• $\bullet$

Joint concavity

We will need the joint concavity of a function on $(\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R}))$. A differentiable function $b$ defined on $(\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R}))$ is concave, if for every $(x^{\prime},m^{\prime})$ and $(x,m)\in(\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R}))$, we have

[TABLE]

whenever $\tilde{X},\tilde{X}^{\prime}\in L^{2}(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}};\mathbb{R})$ with laws $m$ and $m^{\prime}$ respectively.

4 Necessary maximum principle

In this section, we establish a maximum principle of necessary type.

For this end, we assume that $\mathcal{U}$ is a closed convex set (and hence $\mathcal{A}_{\mathbb{F}}$ is convex). Now for a given $u^{\ast}\in\mathcal{A}_{\mathbb{F}}$ and an arbitrary but fixed control $u\in\mathcal{A}_{\mathbb{F}}$, we define

[TABLE]

Note that, thanks to the convexity of $\mathcal{A}_{\mathbb{F}}$, $u^{\theta}\in\mathcal{A}_{\mathbb{F}},\text{for all \hskip 2.84544pt}\theta\in\left[0,1\right]$. We denote by $X^{\theta}:=X^{u^{\theta}}$ and by $X^{\ast}:=X^{u^{\ast}}$ the controlled state processes corresponding to $u^{\theta}$ and $u^{\ast}$ respectively.

For $u^{\ast}\in\mathcal{A}_{\mathbb{F}}$ and the associated controlled state process $X^{\ast}$, let $Y(t):=\frac{d}{d\theta}X^{\theta}(t)|_{\theta=0}$, hence $Y$ satisfies the following SDDE :

[TABLE]

where we used the following notations:

[TABLE]

and $(\tilde{X},\tilde{Y},\tilde{u})$ is an independant copy of $(X,Y,u)$ defined on some probability space $(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})$ and $\tilde{\mathbb{E}}$ denotes the expectation on $(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})$.

Remark 4.1**.**

From the definition of the tilde random variables and since $\sigma$ is deterministic, we have

[TABLE]

*Note that using the previous notations, $\mathbb{\tilde{E}}[\partial_{m}b^{\ast}(t)(\tilde{X}^{\ast}(t))\tilde{Y}(t)]$ is a function of the random vector $(X^{\ast}(t),X^{\ast}({t-\delta}),u^{\ast}(t))$ as it stands for

$\mathbb{\tilde{E}}[\partial_{m}b(t,x,\bar{x},M^{\ast}(t),M^{\ast}(t-\delta),u)(\tilde{X}^{*}(t))\tilde{Y}(t)]|_{x=X^{\ast}(t),\bar{x}=X^{\ast}(t-\delta),u=u^{\ast}(t)}$.*

We assume that the derivative process $Y$ exists and belongs to $L^{2}(\Omega\times[0,T])$ and that the function $\psi_{\delta}^{*}:t\mapsto\mathbb{\tilde{E}}[\partial_{m}\sigma^{*}(t)(\tilde{X}^{*}(t))\tilde{Y}(t)]+\mathbb{\tilde{E}}[\partial_{\bar{m}}\sigma^{*}(t)(\tilde{X}^{*}(t-\delta))\tilde{Y}(t-\delta)]$ is in $\mathbb{L}_{H}^{2}([0,T])$, the integral with respect to the fBm is therefore well defined in the Wiener sense.

Now if $u^{\ast}$ is an optimal control, we have $J(u^{\ast})\leq J(u^{\theta})$, for all $\theta\in\left[0,1\right]$, i.e.,

[TABLE]

with

[TABLE]

where we have used the simplified notations :

[TABLE]

In order to determine the adjoint backward equation associated to $(u^{*},X^{*})$, we suppose that it has in general the following form

[TABLE]

where $(p^{*},q^{*})$ is assumed to be in $\mathcal{L}_{H}^{1,2}([0,T])\times\mathcal{L}_{H}^{1,2}([0,T])$, the integral with respect to the fBm is a fractional Wick-Itô-Skorohod integral and $\alpha$ is some $\mathbb{F}$-adapted process which we have to determine.

Applying the integration by parts formula of Proposition 3.1, to $p^{*}\left(t\right)$ and $Y\left(t\right)$, we obtain

[TABLE]

where we assumed that $Y(t)q^{*}(t)\in\mathcal{L}_{H}^{1,2}([0,T])$ and $\psi^{*}_{\delta}(t)p^{*}(t)\in\mathcal{L}_{H}^{1,2}([0,T])$. By Fubini’s theorem, remark (4.1), remplacing $\psi^{*}_{\delta}(t)$ by its value and by a change of variables using the fact that $Y(t)=0$ for all $t\in[-\delta,0]$ and , we get

[TABLE]

where $(\tilde{p}^{*},\tilde{q}^{*})$ is an independant copy of $(p^{*},q^{*})$ defined on some probability space $(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})$.

By substituting $\left(\ref{PY}\right)$ in $\left(\ref{J}\right)$ and using the terminal value of the BSDE, we get

[TABLE]

Letting the integrand which contains $Y(t)$ equal to zero, we get

[TABLE]

where, for simplicity of notations, we have set :

[TABLE]

We define now the Hamiltonian associated to our problem by:

[TABLE]

by

[TABLE]

where $\mathcal{R}$ is the set of continuous functions from $[0,T]$ into $\mathbb{R}$.

For $u\in\mathcal{A}_{\mathbb{F}}$ with corresponding solution $X=X^{u}$, define, whenever solutions exist, $(p,q):=(p^{u},q^{u})$, by the adjoint equation, in terms of the Hamiltonian, as follows:

[TABLE]

We assume that $(p,q)$ is in $\mathcal{L}_{H}^{1,2}([0,T])\times\mathcal{L}_{H}^{1,2}([0,T])$, the integral with respect to the fBm is understood in the fractional Wick-Itô-Skorohod sense.

For simplicity of notations, we have put :

[TABLE]

Remark 4.2**.**

By the definition of the Hamiltonian given above, the time advanced BSDE (11) is expressed as a first part which appears to be linear and a second part with coefficients that contain the laws or more precisely the joint distribution of the solution processes. This type of Backward Stochastic Differential Equations was never been studied before. However, when there is no mean-field terms and no advance in time, several resolutions were proposed see for instance [5], [7] and for a more general setting [16]. In the section devoted to the applications, we suggest some dynamics where the mean-field term appears mainly in the terminal cost functional, the resolution of the BSDE (11) is in this case possible following a recursive procedure.

We establish in the following theorem the necessary condition of optimality.

Theorem 4.1** (Necessary condition of optimality).**

We assume that the couple $(u^{\ast},X^{\ast})$ is optimal. Suppose that there exists $p^{*}(t)$, $q^{*}(t)$, solutions of the adjoint equation $\left(\ref{eq:2}\right)$ associated to the pair $(u^{\ast},X^{\ast})$ and that $Y(t)q^{*}(t)\in\mathcal{L}_{H}^{1,2}([0,T])$ and $\psi^{*}_{\delta}(t)p^{*}(t)\in\mathcal{L}_{H}^{1,2}([0,T])$. Then

[TABLE]

Proof.

Suppose that $u^{\ast}$ is an optimal control, then we have

[TABLE]

Applying again the integration by parts formula of Proposition 3.1 to $p^{*}\left(t\right)$ and $Y\left(t\right)$ then using Fubini’s theorem, remplacing $\psi^{*}_{\delta}(t)$ by its value, and using a change of variable and the fact that $Y(t)=0$ for all $t\in[-\delta,0]$, we get

[TABLE]

where $A_{1}:=\mathbb{E}[p^{*}(T)Y(T)]$, then by (12), (4) and applying the definition of the Hamiltonian, we get the desired result, that is

[TABLE]

∎

5 Sufficient maximum principle

In this section, we proove sufficient stochastic maximum principle.

Theorem 5.1** (Sufficient condition of optimality).**

Let $u^{\ast}\in\mathcal{A}_{\mathbb{F}}$ with corresponding controlled state $X^{\ast}$. Suppose that there exist $p^{*}(t)$, $q^{*}(t)$ solution of the associated adjoint equation $\left(\ref{eq:2}\right)$. Assume the following:

$X^{u}(t)q^{*}(t)\in\mathcal{L}_{H}^{1,2}([0,T])$, $p^{*}(t)\sigma(t,M(t),M(t-\delta))\in\mathcal{L}_{H}^{1,2}([0,T])$ $\forall u\in\mathcal{A}_{\mathbb{F}}$. 2. 2.

(Concavity) The functions

[TABLE]

are concave for each $t\in[0,T]$ almost surely. 3. 3.

(Maximum condition)

$H(t,X^{*}(t),X^{*}(t-\delta),M^{*}(t),M^{*}(t-\delta),u^{*}(t),p^{*}(t),q^{*}(.))=\hskip 56.9055pt\underset{u\in\mathcal{U}}{\text{ }\sup}H(t,X^{*}(t),X^{*}(t-\delta),M^{*}(t),M^{*}(t-\delta),u,p^{*}(t),q^{*}(.))$**

for all $t\in[0,T]$ almost surely.

Then $(u^{\ast},X^{\ast})$ is an optimal couple for our problem.

Proof.

Let $u\in\mathcal{A}_{\mathbb{F}}$ be a generic admissible control, and $X=X^{u}$ the corresponding controlled state process. By the definition of the performance functional $J$ given by $\left(\ref{P}\right)$, we have

[TABLE]

where

[TABLE]

Applying the definition of the Hamiltonian $\left(\ref{ham}\right)$, we have

[TABLE]

where we used the following notations

[TABLE]

Now using the concavity of $g$ and the terminal value of the BSDE (11) associated to $(u^{*},X^{*})$, we get by Fubini’s theorem

[TABLE]

Applying the integration by parts formula to $p^{*}(t)$ and $X^{\prime}(t)$, we get

[TABLE]

Note that by the change of variables $r=t+\delta$, we have

[TABLE]

where we used that $\mathbb{E}[\int_{0}^{\delta}{X}^{\prime}(r-\delta)\partial_{\bar{x}}H^{*}(r)dr]=\mathbb{E}[\int_{-\delta}^{0}{X}^{\prime}(u)\partial_{\bar{x}}H^{*}(u+\delta)du]=0$ since ${X}^{\prime}(u)=0$ for all $u\in[-\delta,0]$, because $X^{*}(t)=X(t)=x_{0}(t)$ for all $t\in[-\delta,0]$.

Similarly, we get using the previous argument and by Fubini’s theorem

[TABLE]

Hence, we have

[TABLE]

By (14), (15) and (5), we obtain

[TABLE]

due to the concavity assumption on $H$ and because $u^{*}$ satisfies the maximum condition, the first order derivative in $u^{*}$ vanishes. ∎

6 Applications

The main applications of mean-field dynamics that appear in the literature rely mainly on a dependence upon the probability measures through functions of scalar moments of the measures. More precisely, we assume that:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

for some scalar functions $\psi_{1}$, $\psi_{2}$, $\phi_{1}$, $\phi_{2}$, $\gamma_{1}$, $\gamma_{2}$, $\gamma_{3}$ with at most quadratic growth at $\infty$. The functions $\hat{b}$, $\hat{f}$ are defined on $[0,T]\times\mathbb{R}\times\mathbb{R}\times\mathbb{R}\times\mathbb{R}\times\mathcal{U}$, the function $\hat{\sigma}$ is defined on $[0,T]\times\mathbb{R}\times\mathbb{R}$ and $\hat{g}$ is defined on $\mathbb{R}\times\mathbb{R}$. The notation $(\psi,m)$ denotes the integral of the function $\psi$ with respect to the probability measure $m$. The Hamiltonian that we defined in the previous section takes now the following form:

[TABLE]

According to the definition of the differentiability with respect to functions of measures recalled in the preliminaries, the derivative of the Hamiltonian with respect to the measure $m$ for instance, is computed as follows :

[TABLE]

The terminal value of the adjoint BSDE(11) which is $p(T)=\partial_{x}g(T)+\mathbb{\tilde{E}}[\partial_{m}\tilde{g}(T)({X}(T))]$, can be written in terms of the derivatives of the function $\hat{g}$ as follows:

[TABLE]

where $\tilde{X}_{T}$ is an independant copy of $X_{T}$. We study in the following two applications that illustrate the previous results.

6.1 Optimal consumption from a cash flow with delay

We consider the problem of an optimal consumption with a cash flow with delay $X:=X^{\rho}$ given by

[TABLE]

where $\rho$ is the relative consumption rate (our control), $x_{0}$ a bounded deterministic function, $\delta$ a strictly positive constant and $\beta$ is a given deterministic function in $\mathbb{L}_{H}^{2}([0,T])$. The integral with respect to the fBm is therefore a Wiener type integral.

The problem we consider is to find the consumption rate $\rho^{*}$ such that

[TABLE]

where

[TABLE]

over the set $\mathcal{\bar{A}}_{\mathbb{F}}$ of admissible controls which are $\mathbb{F}$-adapted processes with values in $\mathbb{R}^{*}_{+}$, $\xi_{1}>0$ is a given bounded $\mathcal{F}_{T}$-measurable random variable assumed to be in ${\mathcal{L}}_{H}^{1,2}([0,T])$, we also assume that $X$ exists and belongs to $L^{2}(\Omega\times[0,T])$.

Note that the running cost functional we consider in this example is the function $\rho(t)\mapsto\log(\rho(t))$ which is a utility function. Moreover, in order to control the fluctuations of the terminal time-value of the cash flow $X^{\rho}_{T}$, we chose to introduce its mean in the terminal cost functional. Therefore according to the notations used in the previous paragraph, the terminal cost functional is of mean-field type, more precisely it has the following form:

[TABLE]

where $\hat{g}_{\xi_{1}}(x^{\prime})=\xi_{1}x^{\prime}$, therefore $\hat{g}^{\prime}_{\xi_{1}}(x^{\prime})=\xi_{1}$ a.s.

The Hamiltonian of this control problem is given by :

[TABLE]

where $(p,q)$ is the solution of the associated adjoint BSDE

[TABLE]

Inspired by the resolution of the linear BSDE driven by a fractional Brownian motion with Hurst parameter $H>1/2$ done in [5] and [16], we propose a resolution of the anticipated BSDE (18) by solving a sequence of linear BSDEs following this procedure :

**Step 1 ** If $t\in[T-\delta,T]$, the previous BSDE takes the form

[TABLE]

which has the solution

[TABLE]

where $\hat{\mathbb{E}}$ is the quasi-conditional expectation, see [12] for its definition.

Step 2 : If $t\in[T-2\delta,T-\delta]$ and $T-2\delta>0$, we obtain the BSDE

[TABLE]

this BSDE has an expicit solution given by

[TABLE]

where $\theta(t)=\mathbb{E}[p(t+\delta)|\mathcal{F}_{t}]$ and $p(t+\delta)$ is known by step 1.

We continue like this by induction up to and including step n, where n is such that $T-n\delta\leq 0<T-(n-1)\delta$ and we solve the corresponding BSDE on the time interval $[0,T-(n-1)\delta]$ and we solve the corresponding BSDE on the time interval $[0,T-(n-1)\delta]$.

Maximizing H with respect to $\rho$ gives the following first order condition for an optimal consumption rate $\rho^{*}$ :

[TABLE]

Then if

[TABLE]

We get

[TABLE]

where $p$ satisfies the previous anticipated BSDE.

Theorem 6.1**.**

Let $(p,q)$ be the solution of the BSDE (18) and suppose that (19) holds. Then any optimal consumption rate $\rho^{*}$ satisfies (20) and the corresponding optimal wealth $X^{*}$ is given by equation (17).

6.2 Linear-Quadratique Problem with delay

We consider now a Linear-Quadratique (LQ) model for a controlled process $X=X^{\alpha}$ given by the following delayed stochastic differential equation:

[TABLE]

where $\delta>0$ is a given constant, $\beta_{1}$, $x_{0}$ are given bounded deterministic functions, $\beta_{2}$ is a given deterministic function in $\mathbb{L}_{H}^{2}([0,T])$. The integral with respect to the fBm is therefore a Wiener type integral and $\alpha\in\mathcal{A}_{\mathbb{F}}$ is our control process, the set $\mathcal{A}_{\mathbb{F}}$ are the admissible controls assumed to be square integrable $\mathbb{F}$-adapted processes with real values.

We want to minimize the expected value of $(X_{T}-\mathbb{E}[X_{T}])^{2}$ which is the variance of $X_{T}$ with a minimal average use of energy, measured by the integral $\mathbb{E}[\int_{0}^{T}\alpha^{2}(t)dt]$, more precisely, the performance functional is of the following form:

[TABLE]

Our goal is therefore to find the control process $\alpha^{*}\in\mathcal{A}_{\mathbb{F}}$, such that

[TABLE]

Remark 6.1**.**

Including the variance of the state process in the cost functional in order to keep it small is a way to control its sensitivity with respect to the possible variations of the random events. The form of this cost functional is inspired from [20].

Note that the terminal cost functional of our problem has the following form:

[TABLE]

where $\hat{g}(x,x^{\prime})=-\frac{1}{2}(x-x^{\prime})^{2}$, therefore $\partial_{x}\hat{g}(x,x^{\prime})=-(x-x^{\prime})=-\partial_{x^{\prime}}\hat{g}(x,x^{\prime})$. Therefore the terminal value of the solution of the adjoint BSDE is:

[TABLE]

where we used the fact that $\tilde{X}$ and $X$ have the same distribution.

The Hamiltonian of our control problem takes the following form:

[TABLE]

where $(p,q)$ is the solution of the associated adjoint BSDE:

[TABLE]

where we put $\xi_{2}(T)=-(X_{T}-\mathbb{E}[X_{T}])$, we proceed now as we did in the previous exemple by solving a sequence of linear BSDEs.

The function $\alpha\mapsto H(t,X(t),X(t-\delta),\alpha(t),p(t),q(.))$ is maximal when

[TABLE]

where $p^{*}$ satisfies:

[TABLE]

where $\xi_{2}^{*}(T)=-(X_{T}^{*}-\mathbb{E}[X_{T}^{*}])$. Therefore, we have proved the following theorem.

Theorem 6.2**.**

The optimal control $\alpha^{*}$ of the LQ problem (23) is given by (24), where $(X^{*},p^{*},q^{*})$ solve the couple of systems (21) and (25) of forward-backward stochastic differential equations.

Acknowledgement

The first author is supported by the Erasmus + International Credit mobility between Linnaeus University and Cadi Ayyad University for the academic year 2016-2017. The research of the third author is carried out with the support of the Norwegian Research Council, within the research project Challenges in Stochastic Control, Information and Applications (STOCONINF), project number 250768/F20.