Maximum principle for a stochastic delayed system involving terminal state constraints

TL;DR

This paper develops a stochastic maximum principle for control systems with delays and terminal state constraints, using backward delayed equations and variational methods, with applications to linear-quadratic and economic models.

Contribution

It introduces a novel maximum principle for delayed stochastic systems with terminal constraints, employing a backward delayed system and Ekeland's variational principle.

Findings

Derived a stochastic maximum principle for delayed systems with terminal constraints

Applied the principle to linear-quadratic control and economic production models

Validated the theoretical results through illustrative examples

Abstract

We investigate a stochastic optimal control problem where the controlled system is depicted as a stochastic differential delayed equation; however, at the terminal time, the state is constrained in a convex set. We firstly introduce an equivalent backward delayed system depicted as a time-delayed backward stochastic differential equation. Then a stochastic maximum principle is obtained by virtue of Ekeland's variational principle. Finally, applications to a state constrained stochastic delayed linear-quadratic control model and a production-consumption choice problem are studied to illustrate the main obtained result.

Equations

$$\{\begin{array}{l}dX(t)=b(t,X(t),X(t-\delta ),u(t))dt+\sigma (t,X(t),X(t-\delta ),u(t))dW(t),0\le t\le T;\\ X(t)=\eta (t),-\delta \le t\le 0,\end{array}$$

$$\{\begin{array}{l}dX(t)=b(t,X(t),X(t-\delta ))dt+\sigma (t,X(t),X(t-\delta ))dW(t),0\le t\le T;\\ X(t)=\eta (t),-\delta \le t\le 0,\end{array}$$

$$|b(t,x,y)-b(t,x^{\prime },y^{\prime })|+|\sigma (t,x,y)-\sigma (t,x^{\prime },y^{\prime })|\le D(|x-x^{\prime }|+|y-y^{\prime }|);$$

$$|f(t,y,y_{\delta },z)-f(t,y^{\prime },y_{\delta }^{\prime },z^{\prime }){|}^2\le C(|y-y^{\prime }{|}^2+|y_{\delta }-y_{\delta }^{\prime }{|}^2+|z-z^{\prime }{|}^2),$$

$$\{\begin{array}{l}-dY(t)=f(t,Y(t),Y(t-\delta ),Z(t))dt-Z(t)dW(t),0\le t\le T;\\ Y(T)=\xi ,Y(t)=\phi (t),-\delta \le t<0,\end{array}$$

$$E\left[\underset{0\le t\le T}{\sup }|Y(t){|}^2+{\int }_0^T|Z(t){|}^{2}dt\right]\le CE\left[|\xi {|}^2+{\int }_0^T|f(t,0,0,0){|}^{2}dt\right],$$

$$E\left[\underset{0\le t\le T}{\sup }|Y(t)-Y^{\prime }(t){|}^2+\frac{1}{2}{\int }_0^T|Z(t)-Z^{\prime }(t){|}^{2}dt\right]\le CE[|\xi -{\xi }^{\prime }{|}^2].$$

$$|b(t,x,{\varsigma }_t)-b(t,x^{\prime },{\varsigma }_t^{\prime })|+|\sigma (t,x,{\varsigma }_t)-\sigma (t,x^{\prime },{\varsigma }_t^{\prime })|\le C\left(|x-x^{\prime }|+E^{{\mathcal{F}}_t}[|{\varsigma }_t-{\varsigma }_t^{\prime }|]\right),$$

$$\{\begin{array}{l}dX(t)=b(t,X(t),X(t+\delta ))dt+\sigma (t,X(t),X(t+\delta ))dW(t),0\le t\le T;\\ X(0)=x_0,X(t)=\lambda (t),T<t\le T+\delta ,\end{array}$$

$$\begin{array}{rl}& (i)F(v_{ϵ})\le F(v),\\ & (ii)d(v,v_{ϵ})\le ϵ,\\ & (iii)F(u)+\sqrt{ϵ}d(u,v_{ϵ})\ge F(v_{ϵ}),\forall u\in U.\end{array}$$

$${\mathcal{U}}_{ad}\equiv \{u(\cdot )|u(\cdot )\in L_{\mathbb{F}}^2(0,T;{\mathbb{R}}^{n\times d})\}$$

$$\{\begin{array}{l}dX(t)=b(t,X(t),X(t-\delta ),u(t))dt+\sigma (t,X(t),X(t-\delta ),u(t))dW(t),0\le t\le T;\\ X(t)=\eta (t),-\delta \le t\le 0,\end{array}$$

$$J(u(\cdot ))=E\left[{\int }_0^T\tilde{l}(t,X(t),u(t))dt+\varphi (X(T))\right],$$

$${\mathbf{Problem\ A}:\ \ \ \ }\begin{cases}$Minimize$\ \ \ \ J(u(\cdot))\\ $subject$\ $to$\ u(\cdot)\in\mathcal{U}_{ad};\ X(T)\in K.\end{cases}$$

$$|\sigma (t,x,x^{\prime },u_1)-\sigma (t,x,x^{\prime },u_2)|\ge \alpha |u_1-u_2|.$$

$$u\to \sigma (t,x,x^{\prime },u)$$

$$\{\begin{array}{l}-dX(t)=f(t,X(t),X(t-\delta ),q(t))dt-q(t)dW(t),0\le t\le T;\\ X(t)=\eta (t),-\delta \le t\le 0,\end{array}$$

$$\{\begin{array}{l}-dX(t)=f(t,X(t),X(t-\delta ),q(t))dt-q(t)dW(t),0\le t\le T;\\ X(T)=\xi ,X(t)=\eta (t),-\delta \le t<0,\end{array}$$

$$U=\{\xi |E|\xi {|}^2<\infty ,\xi \in K,a.s.\}.$$

$$J(\xi ):=E\left[{\int }_0^{T}l(t,X(t),q(t))dt+\varphi (\xi )\right],$$

$${\mathbf{Problem\ B}:\ \ \ \ }\begin{cases}$Minimize$\ \ \ \ J(\xi)\\ $subject$\ $to$\ \ \ \xi\in U;\ X^{\xi}(0)=a,\end{cases}$$

$$\begin{array}{rl}f(t)=& f(t,X(t),X(t-\delta ),q(t)),f^{\rho }(t)=f(t,X^{\rho }(t),X^{\rho }(t-\delta ),q^{\rho }(t)),\\ f^{\ast }(t)=& f(t,X^{\ast }(t),X^{\ast }(t-\delta ),q^{\ast }(t)),f_{\phi }^{\ast }(t)=f_{\phi }(t,X^{\ast }(t),X^{\ast }(t-\delta ),q^{\ast }(t)),\end{array}$$

$$d({\xi }^1,{\xi }^2):=(E|{\xi }^1-{\xi }^2{|}^2{)}^{\frac{1}{2}}.$$

$${\xi }^{\rho }:={\xi }^{\ast }+\rho (\xi -{\xi }^{\ast }),0\le \rho \le 1.$$

$$\{\begin{array}{l}-d\hat{X}(t)=[f_x^{\ast }(t)\hat{X}(t)+f_{x_{\delta }}^{\ast }(t)\hat{X}(t-\delta )+f_q^{\ast }(t)\hat{q}(t)]dt-\hat{q}(t)dW(t),0\le t\le T;\\ \hat{X}(T)=\xi -{\xi }^{\ast },\hat{X}(t)=0,-\delta \le t<0.\end{array}$$

$$\begin{array}{r}\underset{\rho \to 0}{\lim }\underset{0\le t\le T}{\sup }E|{\tilde{X}}^{\rho }(t){|}^2=0,\\ \underset{\rho \to 0}{\lim }E{\int }_0^T|{\tilde{q}}^{\rho }(t){|}^{2}dt=0,\end{array}$$

$${\tilde{X}}^{\rho }(t)=\frac{X^{\rho }(t)-X^{\ast }(t)}{\rho }-\hat{X}(t),{\tilde{q}}^{\rho }(t)=\frac{q^{\rho }(t)-q^{\ast }(t)}{\rho }-\hat{q}(t).$$

$$F_{\varepsilon}(\xi)=\left\{|X^{\xi}(0)-a|^{2}+\big{(}\max(0,\phi(\xi)-\phi(\xi^{*})+\varepsilon)\big{)}^{2}\right\}^{\frac{1}{2}}.$$

$$\big{<}h_{1},\widehat{X}(0)\big{>}+h_{0}\big{<}\phi_{x}(\xi^{*}),\xi-\xi^{*}\big{>}\geq 0,$$

$$\begin{array}{rl}& F_{\epsilon }({\xi }^{\ast })=\epsilon ;\\ & F_{\epsilon }(\xi )>0,\forall \xi \in U;\\ & F_{\epsilon }({\xi }^{\ast })\le \underset{\xi \in U}{\inf }{F}_{\epsilon }(\xi )+\epsilon .\end{array}$$

Full text

Keywords: Stochastic differential delayed equation; State constraints; Maximum principle

2010 MSC: 93E20, 60H10.

1 Introduction

In 1990, the nonlinear backward stochastic differential equation (BSDE in short) was introduced by Pardoux and Peng [21]. Until now, it had applications in many fields, such as partial differential equation (see [22]), stochastic control (see [10, 27]) and mathematical finance (see [9]). Meanwhile, BSDE itself has been developed to many different branches, such as BSDE with jumps (see [3, 17, 18]), mean-field BSDE (see [4]), time-delayed BSDE (see [5, 7, 8]), anticipated BSDEs (see [23, 25]) and so on. A lot of works have been done for the control problem of such BSDEs. However, fewer works have been done on the control problems of stochastic delayed systems.

For a stochastic delayed system, Chen and Wu [6] obtained a stochastic maximum principle by virtue of a duality between stochastic differential delayed equations (SDDEs in short) and anticipated BSDEs. Øksendal, Sulem and Zhang [20] studied the optimal control problems for SDDEs with jumps. Yu [28] obtained a maximum principle for SDDEs with random coefficients. A maximum principle of optimal control of SDDEs on infinite horizon was proved in Agram, Haadem and Øksendal [2]. Some other recent developments on stochastic delayed system can be found in Huang, Li and Shi [11], Meng and Shen [19], etc.

To the authors’ knowledge, there has been no result concerning the control problem of a stochastic delayed system with state constraints until now. However, the state constraints of stochastic delayed systems indeed exist in reality. In this paper, the stochastic control problem of a forward delayed system with terminal state constraint is studied. The controlled system is depicted as the following SDDE:

[TABLE]

where $X(T)\in K$, a.s., $K\in\mathbb{R}^{n}$ is a convex set. However, there are two (main) difficulties in this study. The first one is that the control system (1.1) is a delayed system, as stated in [6], which is more complex than the classical case. Another difficulty is the terminal state constraint, which is a sample-wise constraint. As interpreted in Ji and Zhou [15], the stochastic control involving sample-wise state constraints cannot be resolved by the classical theory.

Some recent developed results on state constraints (see [10, 13, 15, 16, 14, 1, 26]) as well as the duality relation between time-advanced stochastic differential equations (SDEs, for short) and time-delayed BSDEs (see [5]) may help us to overcome the above mentioned difficulties. Firstly, an equivalent backward formulation of stochastic delayed system (1.1) is introduced, where $X(T)$ is judged as a control variable. Meanwhile, the state constraint turns out to be a control constraint. However, such a treatment brings us both the advantage and the disadvantage. The advantage is that, in the classical control theory, to manage control constraint is easier than to manage state constraint. The disadvantage is that the initial condition ($X(0)=\eta(0)$) now turns into an additional constraint. To deal with the additional initial constraint, Ekeland’s variational principle is used.

Note that the equivalent backward delayed system is described by a time-delayed BSDE, so the adjoint equation of the time-delayed BSDE via duality relation is an anticipated SDE. Therefore, both the delayed system and the anticipated system are needed in our study. As a routine, the variational procedure is made firstly. Then, by virtue of Ekeland’s variational principle, the variational inequality is got. At last, the necessary condition is derived by applying the duality relationship between the backward delayed controlled system and the anticipated forward adjoint system. There is a good thing that the theory of BSDE and our assumption allow us to make the inverse transformation, so that the optimal control process can be solved by the obtained optimal terminal control. To make our conclusions be directly perceived, we also study two applications. One of them is the stochastic delayed linear quadratic (LQ in short) control model. Moreover, a production and consumption choice optimization problem (see [12]) is also adapted to our case.

We organize this article as follows. Some preliminary results about time-delayed BSDE and anticipated SDE are presented in Section 2. In Section 3, the original control problem of a forward delayed controlled system with terminal state constraint is formulated. Then an equivalent transformation is made to get a backward delayed controlled system. Moreover, a stochastic maximum principle is derived, which presents the required condition of the optimal terminal control. In Section 4, two applications are given.

2 Preliminaries

Denote by $(\Omega,\mathcal{F},\mathbb{F},P)$ a probability space such that $\mathcal{F}_{0}$ includes all $P$-null elements of $\mathcal{F}$ and assume the filtration $\mathbb{F}=\{\mathcal{F}_{t},t\geq 0\}$ is generated by a $d$-dimensional standard Brownian motion $W=\{W(t),t\geq 0\}$. Let $T>0$. And $\delta>0$ is a given finite time delay. We denote the following notations:

• $\bullet$

L^{2}(\mathcal{F}_{t};\mathbb{R}^{n})=\{\xi:\Omega\rightarrow\mathbb{R}^{n}\big{|}\xi is $\mathcal{F}_{t}$-measurable, $E|\xi|^{2}$ $<\infty\}$;

• $\bullet$

$L_{\mathbb{F}}^{2}(0,T;\mathbb{R}^{n})=\{\psi:\Omega\times[0,T]\rightarrow\mathbb{R}^{n}$—$\psi(\cdot)$ is $\mathbb{F}$-measurable process, $\displaystyle E\int_{0}^{T}|\psi(t)|^{2}dt<\infty\}$.

Similarly, we can define $L_{\mathbb{F}}^{2}(0,T;\mathbb{R}^{n\times d})$, $L_{\mathbb{F}}^{2}(-\delta,T;\mathbb{R}^{n})$ and $L_{\mathbb{F}}^{2}(0,T+\delta;\mathbb{R}^{n})$.

Now we recall some useful results for the study of the following sections. Consider the following SDDE:

[TABLE]

where $\eta$ is a given continuous function, which represents the initial path of $X$, and $b:[0,T]\times\mathbb{R}^{n}\times\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$ and $\sigma:[0,T]\times\mathbb{R}^{n}\times\mathbb{R}^{n}\rightarrow\mathbb{R}^{n\times d}$ are given measurable functions satisfying the following condition:

• $\mathbf{(H2.1)}$

There exists a constant $D>0$ such that for all $t\in[0,T]$, $x,x^{\prime},y,y^{\prime}\in\mathbb{R}^{n}$,

[TABLE]

\sup\limits_{0\leq t\leq T}\big{(}|b(t,0,0)|+|\sigma(t,0,0)|\big{)}<+\infty.

Then, from Theorem 2.2 in [6], under (H2.1), SDDE (2.1)has the unique adapted solution $X(\cdot)\in L_{\mathbb{F}}^{2}(-\delta,T;$ $\mathbb{R}^{n})$.

For the time-delayed BSDE, we need the following assumption.

• $\mathbf{(H2.2)}$

Assume that $f:\Omega\times[0,T]\times\mathbb{R}^{n}\times\mathbb{R}^{n}\times\mathbb{R}^{n\times d}\rightarrow\mathbb{R}^{n}$ is $\mathbb{F}$-adapted and for every $y,y_{\delta},y^{\prime},y_{\delta}^{\prime}\in\mathbb{R}^{n},z,z^{\prime}\in\mathbb{R}^{n\times d}$

[TABLE]

where $C>0$ is a constant. Moreover $\displaystyle E\int_{0}^{T}|f(t,0,0,0)|^{2}dt$ $<+\infty$.

The following is the well-posedness of time-delayed BSDE.

Proposition 2.1**.**

Suppose $\xi\in L^{2}(\mathcal{F}_{T};\mathbb{R}^{n})$ and $\varphi(\cdot)$ is a given continuous function. Then under (H2.2), for sufficiently small $\delta>0$, the following time-delayed BSDE

[TABLE]

has the unique adapted solution $(Y(\cdot),Z(\cdot))\in L_{\mathbb{F}}^{2}(-\delta,T;\mathbb{R}^{n})\times L_{\mathbb{F}}^{2}(0,T;\mathbb{R}^{n\times d})$, and it satisfies the following estimate:

[TABLE]

with $C>0$. Furthermore, if $(Y^{\prime}(\cdot),Z^{\prime}(\cdot))$ is the solution to (2.2) with $\xi$ replaced by $\xi^{\prime}$, then

[TABLE]

As we stated in the part of Introduction, the anticipated SDE is necessary in our study. The following is the condition for anticipated SDE.

• $\mathbf{(H2.3)}$

Suppose for each $t\in[0,T]$, $r\in[t,T+\delta]$, $b:\Omega\times\mathbb{R}^{n}\times L^{2}(\mathcal{F}_{r};\mathbb{R}^{n})\rightarrow L^{2}(\mathcal{F}_{t};\mathbb{R}^{n})$, $\sigma:\Omega\times\mathbb{R}^{n}\times L^{2}(\mathcal{F}_{r};\mathbb{R}^{n})\rightarrow L^{2}(\mathcal{F}_{t};\mathbb{R}^{n\times d})$ with

[TABLE]

for every $t\in[0,T]$, $x,x^{\prime}\in\mathbb{R}^{n}$, $\varsigma(\cdot),\varsigma^{\prime}(\cdot)\in L_{\mathbb{F}}^{2}(t,T+\delta;\mathbb{R}^{n})$, $r\in[t,T+\delta]$ with $C>0$. Moreover \sup\limits_{0\leq t\leq T}\big{(}|b(t,0,0)|+|\sigma(t,0,0)|\big{)}<+\infty.

Proposition 2.2**.**

Suppose $x_{0}\in\mathbb{R}^{n}$ and $\lambda(\cdot)\in L_{\mathbb{F}}^{2}(T,T+\delta;\mathbb{R}^{n})$ is a given $\mathbb{F}$-adapted process. Assume (H2.3) holds. Then, if $\delta$ is sufficiently small, the anticipated SDE

[TABLE]

has the unique adapted solution $X(\cdot)\in L_{\mathbb{F}}^{2}(0,T+\delta;\mathbb{R}^{n})$.

The above results can be found in Delong and Imkeller [7] and Chen and Huang [5]. The following is the famous Ekeland’s variational principle.

Proposition 2.3**.**

Suppose $(U,d(\cdot,\cdot))$ is a complete metric space with a function $F(\cdot):U\rightarrow\mathbb{R}$ is proper lower semi-continuous. Then, for every $v\in U$ and $\epsilon>0$ such that $F(v)\leq\inf\limits_{u\in U}F(u)+\epsilon$, there is $u_{\epsilon}\in U$ so that

[TABLE]

3 Main result

We study our main result in this part, i.e., a maximum principle about the optimal control of a stochastic delayed system involving terminal state constraint. It should be pointed out that the time-delayed state of the controlled system is different from the case without delay.

3.1 Problem formulation

Let

[TABLE]

be the set of admissible controls. For every given $u(\cdot)$, for the control system, we consider the past-dependent state $X(\cdot)$ depicted as

[TABLE]

where $\eta$ is a given continuous function, $b:[0,T]\times\mathbb{R}^{n}\times\mathbb{R}^{n}\times\mathbb{R}^{n\times d}\rightarrow\mathbb{R}^{n}$ and $\sigma:[0,T]\times\mathbb{R}^{n}\times\mathbb{R}^{n}\times\mathbb{R}^{n\times d}\rightarrow\mathbb{R}^{n\times d}$ are given measurable functions. Define the cost function as follows:

[TABLE]

where $\widetilde{l}:[0,T]\times\mathbb{R}^{n}\times\mathbb{R}^{n\times d}\rightarrow\mathbb{R}^{n}$ and $\phi:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$ are given measurable functions. We give the following assumptions:

• $\mathbf{(H3.1)}$

The functions $b,\sigma,\widetilde{l},$ $\phi$ are all continuously differentiable in the arguments $(x,x^{\prime},u)$, and their derivatives are all bounded.

• $\mathbf{(H3.2)}$

Denote by $C(1+|x|+|u|)$ and $C(1+|x|)$ the bounds of derivatives of $\widetilde{l}$ in its arguments $(x,u)$ and $\phi$ in its argument $x$, respectively.

Therefore, for every given $u(\cdot)\in\mathcal{U}_{ad}$, under assumptions (H3.1) and (H3.2), Eq. (3.1) admits the unique adapted solution $X(\cdot)\in L_{\mathbb{F}}^{2}(-\delta,T;$ $\mathbb{R}^{n})$.

Denote by $K\in\mathbb{R}^{n}$ a given nonempty convex subset. The goal of our control problem is to solve

[TABLE]

3.2 Time-delayed backward formulation

We now show an equivalent backward system of Problem A. In order to do this, one additional assumption is needed:

$\mathbf{(H3.3)}$ There exists $\alpha>0$, and for each $t\in[0,T]$, $x,x^{\prime}\in\mathbb{R}^{n}$ and $u_{1},u_{2}\in\mathbb{R}^{n\times d}$,

[TABLE]

Note that (H3.1) and (H3.3) imply, for every $(t,x,x^{\prime})\in[0,T]\times\mathbb{R}^{n}\times\mathbb{R}^{n}$, that the following function

[TABLE]

is a bijection on $\mathbb{R}^{n\times d}$. Hence, by letting $q\equiv\sigma(t,x,x^{\prime},u)$, we obtain that there is the inverse $\sigma^{-1}$ satisfying $u=\sigma^{-1}(t,x,x^{\prime},q)$. Then we can rewrite (3.1) as

[TABLE]

where $f(t,x,x^{\prime},q)=-b(t,x,x^{\prime},\sigma^{-1}(t,x,x^{\prime},q))$.

Note that $u\rightarrow\sigma(t,x,x^{\prime},u)$ is a bijection, hence $q(\cdot)$ could be regarded as the control, which is the crucial observation that encourages this method for working out Problem A. Furthermore, by virtue of the theory of BSDE, choosing the terminal state $X(T)$ is equal to choosing $q(\cdot)$. Therefore we innovate the following ‘controlled’ system, which essentially is a time-delayed BSDE:

[TABLE]

where now $\xi$ becomes the ‘control’ and belongs to the following set:

[TABLE]

Moreover, here the equivalent cost function is

[TABLE]

with $l(t,x,q)=\widetilde{l}(t,x,\sigma^{-1}(t,x,x^{\prime},q))$.

Hence, the original Problem A is equivalent to the following problem B:

[TABLE]

where $X^{\xi}(0)=a$ (we denote $a=\eta(0)$ in the following for simplicity) is the solution to Eq. (3.2) at the initial time 0 under $\xi$.

In control theory, it is well known that to solve the control constraint is easer than to solve the state constraint. From now on, since Problem A is equivalent to Problem B, we concentrate on dealing with Problem B. The benefit is that by virtue of $\xi$ becoming a control variable now, a control constraint in Problem B replaces the state constraint in Problem A.

Definition 3.1**.**

For $\xi\in U$ and $a\in\mathbb{R}^{n}$, if the solution to (3.2) suits $X^{\xi}(0)=a$, then we call the random variable $\xi$ feasible. For any given $a$, the collection of every feasible $\xi$ is denoted by $\mathcal{N}(a)$. Moreover, if $\xi^{*}\in U$ gets the minimum value of $J(\xi)$ over $\mathcal{N}(a)$, we call $\xi^{*}$ optimal.

3.3 Variational equation

In the following subsections, we denote the following notations for simplicity:

[TABLE]

where $f_{\varphi}$ denotes the partial derivative of f at $\varphi$ with $\varphi=x,x_{\delta},q$, respectively. In $U$, for $\xi^{1},\xi^{2}\in U$, define a metric by

[TABLE]

Apparently, $(U,d(\cdot,\cdot))$ becomes a complete metric space. Suppose $\xi^{*}$ is optimal and, associated with $\xi^{*}$, the pair $(X^{*}(\cdot),$ $q^{*}(\cdot))$ is the corresponding state processes of Eq. (3.2). Because $U$ is convex, for every $\xi$, the following variational control $\xi^{\rho}$ is also in $U$:

[TABLE]

Denote the solution to Eq. (3.2) associated with $\xi=\xi^{\rho}$ by $(X^{\rho}(\cdot),q^{\rho}(\cdot))$. And denote by $(\widehat{X}(\cdot),\widehat{q}(\cdot))$ the solution of following variational equation:

[TABLE]

Remark 3.2**.**

It is easy to know that (3.4) is a linear time-delayed BSDE. By Proposition 2.1, under conditions (H3.1)-(H3.3), Eq. (3.4) has a unique adapted solution in $L_{\mathbb{F}}^{2}(-\delta,T;\mathbb{R}^{n})\times L_{\mathbb{F}}^{2}(0,T;\mathbb{R}^{n\times d})$.

Lemma 3.3**.**

Under assumptions (H3.1)-(H3.3), one has

[TABLE]

where

[TABLE]

Remark 3.4**.**

Since the proof of Lemma 3.3 above is the same as that of Lemma 3.1 in Chen and Huang [5], for simplicity of presentation, we only present the main result and omit the detailed proof. In fact, it is straightforward to prove Lemma 3.3 by applying Proposition 2.1, Taylor expansion and the Lebesgue dominated convergence theorem.

3.4 Variational inequality

We solve the initial constraint $X^{\xi}(0)=a$ and obtain a variational inequality in this subsection.

Given the optimal $\xi^{*}$, for a constat $\varepsilon>0$, define $F\varepsilon(\cdot):U\rightarrow\mathbb{R}$ as follows:

[TABLE]

Remark 3.5**.**

One can test that the functions $|X^{\xi}(0)-a|^{2}$ and $\phi(\xi)$ are both continuous in their argument $\xi$. Hence, $F_{\varepsilon}$, defined on $U$, is also a continuous function in its argument $\xi$.

Theorem 3.6**.**

Under assumptions (H3.1)-(H3.3), suppose that $\xi^{*}$ is an optimal solution to Problem B, so we have $h_{0}\in\mathbb{R}^{+}$ and $h_{1}\in\mathbb{R}^{n}$ satisfying $|h_{0}|+|h_{1}|\neq 0$, so that for every $\xi\in U$, we have the following variational inequality:

[TABLE]

where $\widehat{X}(0)$ is the solution to Eq. (3.4) at time 0.

Proof.

We can check the following properties by the definition,

[TABLE]

Therefore, from Proposition 2.3 (Ekeland’s variational principle), there exists $\xi^{\varepsilon}\in U$ satisfying:

[TABLE]

For every $\xi\in U$, denote $\xi_{\rho}^{\varepsilon}:=\xi^{\varepsilon}+\rho(\xi-\xi^{\varepsilon}),\ 0\leq\rho\leq 1.$ Let $(X_{\rho}^{\varepsilon}(\cdot),q_{\rho}^{\varepsilon}(\cdot))$ (resp. $X^{\varepsilon}(\cdot),q^{\varepsilon}(\cdot)$) be the solution to (3.2) under $\xi_{\rho}^{\varepsilon}$ (resp. $\xi^{\varepsilon}$), and let $(\widehat{X}^{\varepsilon}(\cdot),\widehat{q}^{\varepsilon}(\cdot))$ be the solution to (3.4) when $\xi^{\varepsilon}$ is replaced by $\xi^{*}$. Hence, applying the item (iii) above, one obtains

[TABLE]

On the other hand, similar to Lemma 3.3, one concludes

[TABLE]

Thus

[TABLE]

which leads to the following expansion:

[TABLE]

Moreover,

[TABLE]

In the next, we study the following two cases for given $\varepsilon>0$.

Case 1: There is $\rho_{0}>0$ so that for every $\rho\in(0,\rho_{0})$,

[TABLE]

We see that

[TABLE]

Now, using $\rho$ to split (3.6) and letting $\rho$ to 0, one has

[TABLE]

where

[TABLE]

Case 2: There is a positive series $\{\rho_{n}\}$ satisfying $\rho_{n}\rightarrow 0$, so that

[TABLE]

From the definition of $F_{\varepsilon}$, for large $n$, $F_{\varepsilon}(\xi_{\rho_{n}}^{\varepsilon})=\left\{|X_{\rho_{n}}^{\varepsilon}(0)-a|^{2}\right\}^{\frac{1}{2}}$. Owing to the continuity of $F_{\varepsilon}(\cdot)$, one has $F_{\varepsilon}(\xi^{\varepsilon})=\left\{|X^{\varepsilon}(0)-a|^{2}\right\}^{\frac{1}{2}}$.

Moreover,

[TABLE]

From (3.6), the same as in Case 1,

[TABLE]

where

[TABLE]

For both cases, in summary, from the definition of $F_{\varepsilon}(\cdot)$, one has $h_{0}^{\varepsilon}\geq 0$ and

[TABLE]

Therefore, there exists a convergent subsequence of $(h_{0}^{\varepsilon},h_{1}^{\varepsilon})$ whose limit is denoted by $(h_{0},h_{1})$.

Due to $d(\xi^{\varepsilon},\xi^{*})\leq\sqrt{\varepsilon}$, we have $\xi^{\varepsilon}\rightarrow\xi^{*},\ \ as\ \ \varepsilon\rightarrow 0.$ Then from the estimate of Proposition 2.1, we see that $\widehat{X}^{\varepsilon}(0)\rightarrow\widehat{X}(0)$ as $\varepsilon\rightarrow 0$. Thus (3.5) holds. The desired result is proved now. ∎

By using similar analysis, when $l(t,x,q)\neq 0$, the following variational inequality can be obtained.

Theorem 3.7**.**

Let (H3.1)-(H3.3) hold. Suppose that $\xi^{*}$ is an optimal solution of Problem B. Then we have $h_{0}\in\mathbb{R}^{+}$, $h_{1}\in\mathbb{R}^{n}$ satisfying $|h_{0}|+|h_{1}|\neq 0$, so that for every $\xi\in U$, we have the variational inequality:

[TABLE]

where $l_{\varphi}^{*}(t)=l_{\varphi}(t,X^{*}(t),q^{*}(t))$ denotes the partial derivative of $l^{*}$ at $\varphi$ with $\varphi=x,q,$ respectively, and $(\widehat{X}(\cdot),\widehat{q}(\cdot))$ is the solution to variation equation (3.4).

3.5 Maximum principle

For the sake of establishing the maximum principle, in this part, as the dual equation of Eq. (3.4), the following equation is introduced:

[TABLE]

Remark 3.8**.**

In Eq. (3.10), $f^{*}_{x_{\delta}}|_{t+\delta}$ represents the value of $f^{*}_{x_{\delta}}$ when $t$ is replaced by $t+\delta$.

Remark 3.9**.**

It is easy to see that (3.10) is a linear time-advanced SDE. By Proposition 2.2, under conditions (H3.1)-(H3.3), Eq. (3.10) admits the unique adapted solution in $L_{\mathbb{F}}^{2}(0,T+\delta;\mathbb{R}^{n})$.

Theorem 3.10**.**

Let (H3.1)-(H3.3) hold. If $\xi^{*}$ is optimal to Problem B with $(X^{*}(\cdot),$ $q^{*}(\cdot))$ being the corresponding state of Eq. (3.2), then we have $h_{0}\in\mathbb{R}^{+}$ and $h_{1}\in\mathbb{R}^{n}$ satisfying $|h_{0}|+|h_{1}|\neq 0$, so that for every $\eta\in U$,

[TABLE]

where $m(\cdot)$ is the solution of Eq. (3.10).

Proof.

By using Itô’s formula to \big{<}m(t),\widehat{X}(t)\big{>}, we obtain

[TABLE]

Therefore,

[TABLE]

with

[TABLE]

Paying attention to the terminal and initial conditions, one derives

[TABLE]

Hence

[TABLE]

From the arbitrariness of $\xi\in U$, for every $\eta\in U$, we have

[TABLE]

∎

Now, we let $\partial K$ represent the boundary of $K$, and denote

[TABLE]

According to Theorem 3.10, we directly deduce the following result.

Corollary 3.11**.**

Assume that the assumptions in Theorem 3.10 hold, then for each $\eta\in K$, we have

[TABLE]

Remark 3.12**.**

By the above study, for the optimal terminal control $\xi^{*}$, we obtain the necessary condition. Note that the previous transformation process and (H3.3) allow us to make the inverse transformation. Therefore, the characterization of the optimal control process $u^{*}(\cdot)$ can be derived by the obtained stochastic maximum principle of the optimal terminal control $\xi^{*}$.

4 Applications of the main result

As stated in the section of Introduction, we study two applications of the main result established above in this section.

4.1 Stochastic delayed LQ control involving terminal state constraints

Stochastic delayed LQ control problem involving terminal state constraints is considered in this subsection. In order to simplify the presentation, we focus on the case $d=n=1$. For the higher dimensional situation, one can deal with it in a similar method without substantial difficulty.

Consider the following state equation:

[TABLE]

with $A_{i},B_{i}\in\mathbb{R},\ i=1,2,3$.

Next, we investigate the cost function independent of the running cost without loss of generality. Therefore, subject to $u(\cdot)\in\mathcal{U}_{ad},\ X(T)\in\mathbb{R}^{+},a.s.$, the goal is to minimize the following cost function:

[TABLE]

Without doubt, Eq. (4.2) is an extraordinary example of Problem A with

[TABLE]

Now we give the backward formulation of problem (4.2). Denote

[TABLE]

Then Eq. (4.1) becomes

[TABLE]

and we can rewrite problem (4.2) as follows:

[TABLE]

where

[TABLE]

By Theorem 3.10, if $\xi^{*}$ is optimal, we have $h_{0}\in\mathbb{R}^{+}$ and $h_{1}\in\mathbb{R}$ satisfying $|h_{0}|+|h_{1}|\neq 0$, so that $\forall\ \eta\in U$,

[TABLE]

in which $m(\cdot)$ is the solution to the following adjoint equation:

[TABLE]

Denote $\Omega_{0}:=\{\omega\in\Omega|\xi^{*}(\omega)=0\}$. Now, the following necessary condition can be deduced owing to the arbitrariness of $\xi$.

[TABLE]

where $m(\cdot)$ is the solution of Eq. (4.6).

4.2 Production-consumption choice optimization problem

By applying the maximum principle established before, we investigate a type of production-consumption choice optimization problem in this subsection. The shape for this issue, as in [6], originates from Ivanov and Swishchuk [12]. For the sake of completeness, let us present the model at length.

We assume an investor is going to invest his money to invent goods, and he could obtain benefits from the goods. We mark the capital of investor, the labor at time $t$ and the rate of consumption by $X(t)$, $A(t)$ and $c(t)\geq 0$, respectively. Based upon the assumption that earning of the production is a function of the total sum of the capital and labor, in order to depict this system, Ramsey [24] introduced the following shape:

[TABLE]

Since in the procedure of investment, in reality, there are some risk and delay, Chen and Wu [6] generalized the model in (4.7) to the following case:

[TABLE]

where $\eta$ is a given continuous function.

However, the rationality of the shape has been questioned as there is no constraint for the terminal capital $X(T)$ on the basis of the hypothesis. In fact, in real situations, sometimes the investor will set a goal (constraint) for the terminal capital $X(T)$ in the investment. Hence we believe that in a concordant model of the production and consumption some constraints for the terminal capital $X(T)$ should be considered, i.e., $X(T)\in Q$, where $Q\in\mathbb{R}^{n}$.

Let us consider the following model which modified (4.7) and (4.8):

[TABLE]

For simplicity, let $n=d=1$. Consider the following hypotheses:

• (1)

The function $f(X(t-\delta),A(t))=KX^{\alpha}(t-\delta)A^{\beta}(t)$, where $K,\alpha,\beta$ are some suitable constants. Moreover, let $\alpha=\beta=1$ and $A(t)\equiv y$ be a constant.

• (2)

The terminal constraint $Q\in\mathbb{R}$ is a given convex set.

Under the above assumptions, we can rewrite our shape as follows:

[TABLE]

By electing the hypothesis rate $c(t)\geq 0$ under the terminal constraint $X(T)\in Q$, the purpose is to maximize the following desired function:

[TABLE]

where $r$ represents the bond rate, $\gamma\in(0,1)$, and $1-\gamma$ represents the investor’s risk aversion.

Clearly, this is a special case of Problem A when

[TABLE]

with

[TABLE]

Now, let $q\equiv\sigma(x^{\prime},c)$ and $f(x^{\prime},q)=-Ky\cdot x^{\prime}+\widetilde{\sigma}(x^{\prime},q)$, where $\widetilde{\sigma}$ is the inverse function of $\sigma$ w.r.t $c$, i.e., $c=\widetilde{\sigma}:=\widetilde{\sigma}(x^{\prime},q)$. Then one can rewrite (4.9) as follows:

[TABLE]

Define $U=\{\xi|E|\xi|^{2}<\infty,\ \xi\in Q,\ a.s.\}$ and consider the following performance function:

[TABLE]

Then problem (4.10) is equivalent to the following problem:

[TABLE]

Consider the adjoint equation

[TABLE]

where $h_{1}\in\mathbb{R}$ is a parameter. Denote $\Omega_{0}:=\{\omega\in\Omega|\xi^{*}(\omega)=\partial Q\}$. Therefore, by using Theorem 3.10, one obtains the result below.

Theorem 4.1**.**

Suppose $(X^{*}(\cdot),c^{*}(\cdot))$ is an optimal pair to problem (4.10), then we have $h_{0}\in\mathbb{R}^{+}$ and $h_{1}\in\mathbb{R}$ satisfying $|h_{0}|+|h_{1}|\neq 0$ so that $\xi^{*}\equiv X^{*}(T)$, we have

[TABLE]

where $m(\cdot)$ is the solution to Eq. (4.14) with parameter $h_{1}$.

5 Conclusions

In this content, we study a stochastic optimal control problem for stochastic differential delayed equation with terminal state constraint (at the terminal time, the state is constrained in a convex set). However, the control problem with terminal non-convex state constraint is still open. We will focus on the open problem in the future study.

Competing interests

The first and second authors announce to have no competing interests.

Authors’ contributions

The first and second authors contributed equally to the writing of this paper. Moreover, the first and second authors read and approved the final manuscript.

Acknowledgements

The first author would like to appreciate the Department of Mathematics of University of Central Florida, USA, for its hospitality, and express the gratitude to Prof. Qingmeng Wei for her careful reading of this paper and helpful comments.