A stochastic Pontryagin maximum principle on the Sierpinski gasket

TL;DR

This paper develops a stochastic Pontryagin maximum principle tailored for control problems on the Sierpinski gasket, addressing unique challenges posed by fractal geometry and measure singularity.

Contribution

It introduces a novel stochastic maximum principle for fractal spaces, incorporating two necessity equations due to measure singularity, expanding control theory on complex geometries.

Findings

Derived an order comparison lemma using heat kernel estimates.

Established a Pontryagin maximum principle with two necessity equations.

Analyzed linear regulator problems on the gasket.

Abstract

In this paper, we consider stochastic control problems on the Sierpinski gasket. An order comparison lemma is derived using heat kernel estimate for Brownian motion on the gasket. Using the order comparison lemma and techniques of BSDEs, we establish a Pontryagin stochastic maximum principle for these control problems. It turns out that the stochastic maximum principle on the Sierpinski gasket involves two necessity equations in contrast to its counterpart on Euclidean spaces. This effect is due to singularity between the Hausdorff measure and the energy dominant measure on the gasket, which is a common feature shared by many fractal spaces. The linear regulator problems on the gasket is also considered as an example.

Equations

$$\left\{\begin{aligned} \mathcal{E}(u,v)&=\lim_{m\to\infty}\mathcal{E}^{(m)}(u,v),\;\;u,v\in\mathcal{F}(\mathbb{S}),\\ \mathcal{F}(\mathbb{S})&=\Big{\{}u\in\ell(\mathbb{S}):\lim_{m\to\infty}\mathcal{E}^{(m)}(u,u)<\infty\Big{\}},\end{aligned}\right.$$

$$\mathcal{E}^{(m)}(u,v)=\Big{(}\frac{5}{3}\Big{)}^{m}\sum_{x,y\in\mathrm{V}_{m}:|x-y|=2^{-m}}[u(x)-u(y)][v(x)-v(y)],\;\;u,v\in\ell(\mathrm{V}_{m}).$$

$$M_t^{[u]}={\int }_0^t\zeta (X_r)d{W}_r,\text{for all}t\ge 0,$$

$$\sup_{x\in\mathbb{S}}\mathbb{E}_{x}\big{(}f(X_{t})e^{\kappa\langle W\rangle_{t}}\big{)}\leq\max\{1,t^{-d_{s}/2}\}\|f\|_{L^{1}(\mu)}\mathrm{E}_{\gamma_{s},\gamma_{s}}[C_{\ast}\kappa\max\{t,t^{\gamma_{s}}\}],$$

$$J(u)\triangleq\mathbb{E}_{\lambda}\bigg{(}h(x(T))+\int_{0}^{T}f_{1}(t,x(t),u(t))dt+\int_{0}^{T}f_{2}(t,x(t),u(t))d\langle W\rangle_{t}\bigg{)},$$

$$\{\begin{array}{rl}dx(t)& =b_1(t,x(t),u(t))dt+b_2(t,x(t),u(t))d⟨W{⟩}_t\\ & +\sigma (t,x(t),u(t))d{W}_t,t\in (0,T],{\mathbb{P}}_{\lambda }\text{-a.s.},\\ x(0)& =x_0,\end{array}$$

$$\mathbb{E}_{\lambda}\bigg{(}|h(x(T))|+\int_{0}^{T}|f_{1}(t,x(t),u(t))|dt+\int_{0}^{T}|f_{2}(t,x(t),u(t))|d\langle W\rangle_{t}\bigg{)}<\infty,$$

$${\mathrm{minimize}}_{u\in \mathcal{A}[0,T]}J(u),$$

$${\mathfrak{M}}_1=dt\times {\mathbb{P}}_{\lambda },$$

$$\mathfrak{M}_{2}\big{(}\llbracket\sigma_{1},\sigma_{2}\rrparenthesis\big{)}=\mathbb{E}_{\lambda}\big{(}\langle W\rangle_{\sigma_{2}}-\langle W\rangle_{\sigma_{1}}\big{)},$$

$$\{\begin{array}{rlrl}|\phi (t,x,u)-\phi (t,\hat{x},\hat{u})|& \le M|x-\hat{x}|+\rho (u,\hat{u}),& t& \in [0,T],x,\hat{x}\in \mathbb{R},u,\hat{u}\in \mathbb{U},\\ |\phi (t,0,u)|& \le M,& t& \in [0,T],u\in \mathbb{U},\end{array}$$

$$|{\partial }_x\phi (t,x,u)$$

$$\le M|x-\hat{x}|+\rho (u,\hat{u}),t\in [0,T],x,\hat{x}\in \mathbb{R},u,\hat{u}\in \mathbb{U},$$

$$\{\begin{array}{rl}dp(t)& =-[{\partial }_x{b}_1(t)p(t)-{\partial }_x{f}_1(t)]dt\\ & -[{\partial }_x{b}_2(t)p(t)+{\partial }_x\sigma (t)q(t)-{\partial }_x{f}_2(t)]d⟨W{⟩}_t\\ & +q(t)d{W}_t,t\in [0,T],{\mathbb{P}}_{\lambda }\text{-a.s.},\\ p(T)& =-{\partial }_{x}h(\bar{x}(T)),\end{array}$$

$$\left\{\begin{aligned} dP(t)&=-[2\partial_{x}b_{1}(t)P(t)+\partial_{x}^{2}b_{1}(t)p(t)-\partial_{x}^{2}f_{1}(t)]dt\\ &\quad-\big{[}\big{(}2\partial_{x}b_{2}(t)+\partial_{x}\sigma(t)^{2}\big{)}P(t)+\partial_{x}\sigma(t)Q(t)+\partial_{x}^{2}b_{2}(t)p(t)+\partial_{x}^{2}\sigma(t)q(t)-\partial_{x}^{2}f_{2}(t)\big{]}d\langle W\rangle_{t}\\ &\quad+Q(t)dW_{t},\qquad t\in[0,T],\;\mathbb{P}_{\lambda}\text{-a.s.},\\ P(T)&=-\partial_{x}^{2}h(\bar{x}(T)),\end{aligned}\right.$$

$$H_1(t,x,u)\triangleq b_1(t,x,u)p(t)-f_1(t,x,u),$$

$$H_2(t,x,u)\triangleq b_2(t,x,u)p(t)+\sigma (t,x,u)q(t)-f_2(t,x,u)+\frac{1}{2}[\sigma (t,x,u)-\sigma (t,x,\bar{u}(t)){]}^{2}P(t).$$

$$\{\begin{array}{rlr}{H}_1(t,\bar{x}(t),\bar{u}(t))& =\underset{u\in \mathbb{U}}{\max }{H}_1(t,\bar{x}(t),u),& {\mathfrak{M}}_1\text{-a.e.},\\ H_2(t,\bar{x}(t),\bar{u}(t))& =\underset{u\in \mathbb{U}}{\max }{H}_2(t,\bar{x}(t),u),& {\mathfrak{M}}_2\text{-a.e.},\end{array}$$

$$m_{k,\lambda}(I;E)=\mathbb{E}_{\lambda}\Big{[}\Big{(}\int_{I}1_{E}(t,\omega)d\langle W\rangle_{t}\Big{)}^{k}\Big{]}.$$

$$m_{k+1,\lambda }(I_{ϵ};E)\le C_{\ast }(k+1)|I_{ϵ}{|}^{1-d_s/2}{m}_{k,\lambda }(I_{ϵ};E),\text{for all}k\in {\mathbb{N}}_{+}.$$

$$m_{l,\lambda }(I_{ϵ};E)=\mathrm{o}(m_{k,\lambda }(I_{ϵ};E)),\text{as}ϵ\to 0,$$

$$\mathbb{E}_{\lambda}\Big{[}\Big{(}\int_{0}^{\infty}\phi_{\epsilon}(t)d\langle W\rangle_{t}\Big{)}^{k}\Big{]}=\mathbb{E}_{\lambda}\Big{[}k!\int_{0<t_{1}<\cdots<t_{k}<\infty}\phi_{\epsilon}(t_{1})\cdots\phi_{\epsilon}(t_{k})d\langle W\rangle_{t_{1}}\cdots d\langle W\rangle_{t_{k}}\Big{]}.$$

$$\displaystyle\;\quad\mathbb{E}_{\lambda}\Big{[}\Big{(}\int_{0}^{\infty}\phi_{\epsilon}(t)d\langle W\rangle_{t}\Big{)}^{k+1}\Big{]}$$

$$\displaystyle=\mathbb{E}_{\lambda}\Big{[}(k+1)!\int_{0<t_{1}<\cdots<t_{k+1}<\infty}\phi_{\epsilon}(t_{1})\cdots\phi_{\epsilon}(t_{k+1})d\langle W\rangle_{t_{1}}\cdots d\langle W\rangle_{t_{k+1}}\Big{]}$$

$$\displaystyle=\mathbb{E}_{\lambda}\Big{[}(k+1)!\int_{0<t_{1}<\cdots<t_{k}<\infty}\phi_{\epsilon}(t_{1})\cdots\phi_{\epsilon}(t_{k})\mathbb{E}_{\lambda}\Big{(}\int_{t_{k}}^{\infty}\phi_{\epsilon}(t_{k+1})d\langle W\rangle_{t_{k+1}}\Big{|}\,\mathcal{F}_{t_{k}}^{\lambda}\Big{)}d\langle W\rangle_{t_{1}}\cdots d\langle W\rangle_{t_{k}}\Big{]}.$$

$$\displaystyle\mathbb{E}_{\lambda}\Big{(}\int_{t_{k}}^{\infty}\phi_{\epsilon}(t_{k+1})d\langle W\rangle_{t_{k+1}}\Big{|}\,\mathcal{F}_{t_{k}}^{\lambda}\Big{)}$$

$$={\int }_{t_k}^{\infty }{1}_{I_{ϵ}}(t_{k+1})(P_{t_{k+1}-t_k}\mu )(X_{t_k})d{t}_{k+1},$$

$$P_t\lambda (x)\triangleq {\int }_{\mathbb{S}}{p}_t(x,y)\lambda (dy),x\in \mathbb{S},$$

$$C_{\ast }^{-1}\max \{1,t^{-d_s/2}\}\le p_t(x,y)\le C_{\ast }\max \{1,t^{-d_s/2}\},t\in (0,\infty ),x,y\in \mathbb{S},$$

$$\Vert P_t\mu {\Vert }_{L^{\infty }}\le C_{\ast }\max \{1,t^{-d_s/2}\},t\in (0,\infty ).$$

$$\displaystyle\;\quad\mathbb{E}_{\lambda}\Big{(}\int_{t_{k}}^{\infty}\phi_{\epsilon}(t_{k+1})d\langle W\rangle_{t_{k+1}}\Big{|}\,\mathcal{F}_{t_{k}}^{\lambda}\Big{)}$$

$$\le {\int }_{t_k}^{\infty }{1}_{I_{ϵ}}(t_{k+1})(t_{k+1}-t_k{)}^{-d_s/2}d{t}_{k+1}$$

$$\le {\int }_{t_k}^{t_k+|I_{ϵ}|}(t_{k+1}-t_k{)}^{-d_s/2}d{t}_{k+1}+|I_{ϵ}{|}^{-d_s/2}{\int }_0^{\infty }{1}_{I_{ϵ}}(t_{k+1})d{t}_{k+1}$$

Full text

1 Introduction

Recently, to study non-linear analysis on the Sierpinski gasket, [7] developed a theory of backward stochastic differential equations (BSDEs) on the Sierpinski gasket. BSDEs and related stochastic analysis on fractals, though initially considered as efficient tools to treat quasi-linear parabolic PDEs on fractals, also have interests on their own from a mathematical finance point of view. Several interesting mathematical finance problems are formulated as stochastic control problems on Euclidean spaces, which are based upon the assumption that uncertainties in financial models are sourced from Brownian filtration on Euclidean spaces. However, it had been widely observed from the real data that many financial time series exhibit fractal behaviours (see, for example, [1, 8, 2] and etc.), which suggests the possibility that uncertainties in the markets might come from filtrations exhibiting fractal structures. Therefore, it is of significance to consider stochastic control problems for controlled systems with noise coming from filtrations determined by the diffusions on fractals.

The motivation of this paper is to establish a stochastic Pontryagin maximum principle for stochastic control problems on the Sierpinski gasket, with uncertainties in the controlled dynamic systems generated by the diffusion on the gasket. It turns out that, in contrast to its counterpart on Euclidean spaces, the stochastic maximum principle on the gasket consists of two necessity equations rather than a single one (see [9] and [10, Section 3.2]). As we shall see, this is due to the singularity between two measures which are both necessary for analysis on fractals.

This paper is organized as follows. In Section 2, we introduce notations which will be enforced throughout this paper, and review some related results in literature. The main results of this paper is formulated and collected in Section 3. Section 4 is devoted to the proof of the stochastic maximum principle on the Sierpinski gasket. The linear regulator problem on the gasket is considered in Section 5 as an example. Though results of this paper are established for two-dimensional Sierpinski gasket, we however believe that our results also hold for higher-dimensional cases, where argument in this paper should remain valid.

2 Notations and related results

In this section, we introduce notations which will be enforced throughout this paper. We also review several results in literature needed in the following sections.

Let $\mathrm{V}_{0}=\{p_{1},p_{2},p_{3}\}\subseteq\mathbb{R}^{2}$ with $p_{1}=(0,0),\;p_{2}=(1,0),\;p_{3}=(\tfrac{1}{2},\tfrac{\sqrt{3}}{2})$, and $\mathrm{F}_{i}:\mathbb{R}^{2}\to\mathbb{R}^{2},\;i=1,2,3$ be the contraction mappings given by $\mathrm{F}_{i}(x)=\tfrac{1}{2}(x+p_{i}),\;x\in\mathbb{R}^{2},\;i=1,2,3.$ Define $\mathrm{V}_{m},\;m\in\mathbb{N}$ inductively by $\mathrm{V}_{m+1}=\mathrm{F}_{1}(\mathrm{V}_{m})\cup\mathrm{F}_{2}(\mathrm{V}_{m})\cup\mathrm{F}_{3}(\mathrm{V}_{m}),\;m\in\mathbb{N}$, and $\mathrm{V}_{\ast}=\bigcup_{m=0}^{\infty}\mathrm{V}_{m}$. The (two-dimensional) Sierpinski gasket is defined to be the closure $\mathbb{S}=\bar{\mathrm{V}}_{\ast}$ of $\mathrm{V}_{\ast}$ in $\mathbb{R}^{2}$.

For a given set $\mathrm{V}$, we denote by $\ell(\mathrm{V})$ the space of all real-valued functions on $\mathrm{V}$. The standard Dirichlet form $(\mathcal{E},\mathcal{F}(\mathbb{S}))$ on the Sierpinski gasket $\mathbb{S}$ is defined by

[TABLE]

where the forms $\mathcal{E}^{(m)},\;m\in\mathbb{N}$ are given by

[TABLE]

Let $\nu$ be the Hausdorff measure on $\mathbb{S}$ with weight $(\frac{1}{3},\frac{1}{3},\frac{1}{3})$, that is, $\nu$ is the unique Borel probability measure on $\mathbb{S}$ such that $\nu(\mathbb{S}_{[\omega]_{m}})=3^{-m}$ for each $\omega\in\tilde{\Omega}$ and each $m\in\mathbb{N}$. Then the form $\mathcal{E}$ is a regular Dirichlet form on $L^{2}(\mathbb{S};\nu)$, and $\mathcal{F}(\mathbb{S})$ is the corresponding Dirichlet space.

The Kusuoka measure $\mu$ on $\mathbb{S}$ is defined by $\mu=(\mu_{1}+\mu_{2}+\mu_{3})/3$, where $\mu_{i}$ is the energy measures of the harmonic function with boundary value $1_{p_{i}}$, which is the unique minimizer of $\inf\{\mathcal{E}(u,u):u\in\mathcal{F}(\mathbb{S})\allowbreak\;\text{and}\allowbreak\;u|_{\mathrm{V}_{0}}=1_{p_{i}}\}$.

According to the general theory of Dirichlet forms and Markov processes (see [4, Chapter 7]), associated to the form $(\mathcal{E},\mathcal{F}(\mathbb{S}))$ there exists a standard Hunt process \mathbf{M}=\big{(}\Omega,\mathcal{F},\{X_{t}\}_{t\in[0,\infty]},\allowbreak\{\mathbb{P}_{x}\}_{x\in\mathbb{S}\cup\{\Delta\}}\big{)} with state space $\mathbb{S}$, where $\Delta$ is the “cemetery” of $\mathbf{M}$. The process $\{X_{t}\}_{t\geq 0}$ is called *Brownian motion *on $\mathbb{S}$. The semigroup of $\{X_{t}\}_{t\geq 0}$ will be denoted by $\{P_{t}\}_{t\geq 0}$.

Let $\mathcal{P}(\mathbb{S})$ be the family of all Borel probability measures on $\mathbb{S}$. For each $\lambda\in\mathcal{P}(\mathbb{S})$, the probability measure $\mathbb{P}_{\lambda}$ on $\Omega$ is defined by $\mathbb{P}_{\lambda}(A)=\int_{\mathbb{S}}\mathbb{P}_{x}(A)\lambda(dx),\;A\in\mathcal{F}$. The expectation with respect to $\mathbb{P}_{\lambda}$ will be denoted by $\mathbb{E}_{\lambda}$. Let $\mathcal{F}_{t}^{0}=\sigma\left(X_{r}:r\leq t\right),\;t\geq 0$, $\mathcal{F}_{t}^{\lambda}$ the $\mathbb{P}_{\lambda}$-completion of $\mathcal{F}_{t}^{0}$ in $\mathcal{F}$, and $\{\mathcal{F}_{t}\}_{t\geq 0}$ the minimal completed admissible filtration (cf. [4, p. 385]) of $\{X_{t}\}_{t\geq 0}$, that is, $\mathcal{F}_{t}=\bigcap_{\lambda\in\mathcal{P}(\mathbb{S})}\mathcal{F}_{t}^{\lambda},\;t\geq 0$.

We end this section with a review on the representing martingale on the Sierpinski gasket. The following result was first shown in [6, Theorem (5.4)] (see also [7, Theorem 2.6]).

Theorem 2.1**.**

There exists a martingale additive functional $W_{t}$ satisfying the following:

(i) $W_{t}$ has $\mu$ as its energy measure;

(ii) For any $u\in\mathcal{F}(\mathbb{S})$, there exists a unique $\zeta\in L^{2}(\mathbb{S};\mu)$ such that

[TABLE]

where $M^{[u]}$ is the martingale part of $u(X_{t})-u(X_{0})$.

The martingale additive functional $W$ given by (2.1) is called the Brownian martingale on $\mathbb{S}$. The following result on the singularity between the Lebesgue-Stieltjes measure induced by $t\mapsto\langle W\rangle_{t}$ and the Lebesgue measure on $[0,\infty)$ was proved in [7, Lemma 4.10].

Lemma 2.2**.**

The Lebesgue-Stieltjes measure $d\langle W\rangle_{t}(\omega)$ is singular to the Lebesgue measure $dt$ on $[0,\infty)$ $\mathbb{P}_{\nu}\mbox{-a.e.}\;\omega\in\Omega$.

The following lemma, which is shown in [7, Lemma 4.11], gives the exponential integrability of $\langle W\rangle_{t}$.

Lemma 2.3**.**

For each $f\in L_{+}^{1}(\mu)$ and $\kappa,t>0$,

[TABLE]

where $C_{\ast}>0$ is a universal constant.

3 Formulation of the main result

Let $\lambda\in\mathcal{P}(\mathbb{S})$ satisfy $\lambda\ll\nu$. Let the decision space $(\mathbb{U},\rho)$ be a separable metric space. Let $h:\mathbb{R}\to\mathbb{R},\;\allowbreak f_{1}:[0,T]\times\mathbb{R}\times\mathbb{U}\to\mathbb{R},\;\allowbreak f_{2}:[0,T]\times\mathbb{R}\times\mathbb{U}\to\mathbb{R}$ be Borel measurable functions. For any $\mathbb{U}\text{-}$valued progressively measurable process $u(t)$, we introduce the *cost functional *

[TABLE]

for the controlled system $x(t)$ of which the dynamics is given by the following SDE on \big{(}\Omega,\mathcal{F},\{\mathcal{F}_{t}^{\lambda}\}_{t\geq 0},\mathbb{P}_{\lambda}\big{)}:

[TABLE]

where $\varphi:[0,T]\times\mathbb{R}\times\mathbb{U}\to\mathbb{R},\;\varphi=b_{1},b_{2},\sigma$ are Borel measurable functions, and $x_{0}\in\mathcal{F}_{0}^{\lambda}$.111The existence and uniqueness of solutions to (3.2) can be easily shown by an a priori estimate similar to [7, eqn. (3.8), p. 8].

Definition 3.1**.**

Denote by $\mathcal{A}[0,T]$ the family of all $\mathbb{U}\text{-}$valued processes $u(t)$ such that

[TABLE]

where $x(t)$ is the controlled process given by (3.2). Any $u\in\mathcal{A}[0,T]$ is called an admissible control, and $(x(\cdot),u(\cdot))$ is called an admissible pair.

We consider the following optimization problem

[TABLE]

subject to the controlled dynamics (3.2). To formulate our result, we shall need the following definition.

Definition 3.2**.**

We define the measure $\mathfrak{M}_{1}$ on $[0,\infty)\times\Omega$ to be

[TABLE]

and the measure $\mathfrak{M}_{2}$ to be the unique measure on the optional $\sigma\text{-}$field222That is, the $\sigma\text{-}$field on $[0,\infty)\times\Omega$ generated by the family of all right continuous left limit processes. on $[0,\infty)\times\Omega$ such that

[TABLE]

for any $\{\mathcal{F}_{t}\}\text{-}$stopping times $\sigma_{1},\sigma_{2}$ with $\sigma_{1}\leq\sigma_{2}$, where $\llbracket\sigma_{1},\sigma_{2}\rrparenthesis=\{(t,\omega)\in[0,\infty)\times\Omega:\sigma_{1}(\omega)\leq t<\sigma_{2}(\omega)\}$.

Remark 3.3*.*

By $\lambda\ll\nu$ and Lemma 2.2, the measures $\mathfrak{M}_{1}$ and $\mathfrak{M}_{2}$ are mutually singular.

Theorem 3.4**.**

*Let $\lambda\in\mathcal{P}(\mathbb{S})$ be absolutely continuous with respect to $\nu$. Assume that:

(A.1)*

[TABLE]

*for $\varphi=b_{1},b_{2},\sigma,f_{1},f_{2},h$, and

(A.2)*

[TABLE]

for $\varphi=b_{1},b_{2},\sigma,f_{1},f_{2},h$, where $M>0$ is a constant.

Suppose that $(\bar{x}(\cdot),\bar{u}(\cdot))$ is a solution to (P). Let $(p(\cdot),q(\cdot))$ and $(P(\cdot),Q(\cdot))$ be the solutions of the adjoint equations

[TABLE]

and

[TABLE]

and let $H_{1}(t,x,u),\;H_{2}(t,x,u)$ be the Hamiltonians defined by

[TABLE]

[TABLE]

Then

[TABLE]

Remark 3.5*.*

(i) Notice that the assumptions (A.1) and (A.2) imply that $\varphi,\partial_{x}\varphi,\partial_{x}^{2}\varphi$ are uniformly bounded for $\varphi=b_{1},b_{2},\sigma,f_{1},f_{2},h$. Indeed, the assumption (A.1) implies the uniform boundedness of $\varphi$. The boundedness of $\partial_{x}\varphi(t,0,u)$ for $(t,u)$ can also be deduced from (A.1) with $\hat{u}=u$, which together with (A.2) implies the uniform boundedness of $\partial_{x}\varphi$. Similarly, $\partial_{x}^{2}\varphi$ is also uniformly bounded.

(ii) The adjoint equations (3.6) and (3.7) are introduced in order to reduce the general case with a non-trivial $h(x(T))$ in the cost functional $J(u)$ to the one without an $h(x(T))$ term. In other words, it transforms the cost $h(x(T))$ at terminal time into a cumulative cost over the interval $[0,T]$. This can be seen more clearly from the proof of Theorem 3.4.

4 Proof of the stochastic maximum principle

In this section, we prove Theorem 3.4 for the optimization problem (P) on the Sierpinski gasket. Our argument is based on the idea of approximation and duality used in the paper [9] and the monograph [10] for classical Euclidean setting, while overcoming some difficulties concerning the driver martingale $W$ on the Sierpinski gasket. More specifically, as we shall see, a crucial ingredient of our argument is an order comparison lemma (Lemma 4.2), which is needed for stochastic Taylor expansions. Another technical lemma crucial to the proof of Theorem 3.4 is Lemma 4.6, which gives the orders of approximation errors.

Definition 4.1**.**

Let $\lambda\in\mathcal{P}(\mathbb{S}),\,k\geq 1$ and $E\in\mathcal{B}([0,\infty)\times\Omega)$ be a progressively measurable set. For each $I\in\mathcal{B}([0,\infty))$, we denote

[TABLE]

Clearly, the map $I\mapsto|I|+m_{1,\lambda}(I;\Omega)$ is a Borel measure on $\mathcal{B}([0,\infty))$, where $|\cdot|$ is the one-dimensional Lebesgue measure. We denote by $\mathcal{B}_{\lambda}([0,\infty))$ the completion of $\mathcal{B}([0,\infty))$ with respect to the measure $|\cdot|+m_{1,\lambda}(\cdot\,;\Omega)$.

Lemma 4.2**.**

Let $\lambda\in\mathcal{P}(\mathbb{S})$, and $E\in\mathcal{B}([0,\infty)\times\Omega)$ be a progressively measurable set. Let $\{I_{\epsilon}\}_{\epsilon>0}$ be a family of $\mathcal{B}_{\lambda}([0,\infty))$-measurable subsets of $[0,\infty)$ such that $\lim_{\epsilon\to 0}|I_{\epsilon}|=0$. Then, for some universal constant $C_{\ast}>0$,

[TABLE]

In particular,

[TABLE]

for all $k\in\mathbb{N}_{+}$ and $l>k$.

Proof.

Let $\phi_{\epsilon}(t)=\phi_{\epsilon}(t,\omega)=1_{I_{\epsilon}}(t)1_{E}(t,\omega)$. Then, for each $\epsilon>0$, $\phi_{\epsilon}$ is a bounded progressively measurable process. Clearly, we have the following iterated integral representation

[TABLE]

Since $\phi_{\epsilon}$ is progressively measurable, we have $\phi_{\epsilon}(t)\in\mathcal{F}_{t}^{\lambda}$. Therefore, by (4.3) and the tower property,

[TABLE]

Recall that $\phi_{\epsilon}(t,\omega)\leq 1_{I_{\epsilon}}(t)$. By [7, Lemma 4.17], we have

[TABLE]

where, for any Borel measure $\lambda$ on $\mathbb{S}$,

[TABLE]

with $p_{t}(x,y)$ being the transition kernel of $\{X_{t}\}_{t\geq 0}$, which is jointly continuous on $\mathbb{S}\times\mathbb{S}$. By [5, Theorem 5.3.1], there exists a universal constant $C_{\ast}>0$ such that

[TABLE]

where $d_{s}=2\log 3/\log 5<2$ is the spectral dimension of $\{X_{t}\}_{t\geq 0}$. Therefore,

[TABLE]

For $I_{\epsilon}$ with $|I_{\epsilon}|\leq 1$, by (4.5),

[TABLE]

Hence, by (4.4),

[TABLE]

By (4.3) again, we conclude that

[TABLE]

which is (4.1).

When $l$ is an integer, the asymptotic (4.2) is a direct corollary of (4.1). For real-valued $l>k$, the conclusion follows easily from interpolation

[TABLE]

∎

Remark 4.3*.*

The order estimate (4.1) implies that $m_{k,\lambda}(I_{\epsilon};E)=\mathrm{O}(|I_{\epsilon}|^{k(1-d_{s}/2)})$, which is quite sharp. In fact, since the heat kernel estimate (4.6) is two-sided, by [7, Lemma 4.17], we have that

[TABLE]

Therefore, by (4.1),

[TABLE]

Notice that the reverse of the above inequality is a direct consequence of Hölder’s inequality. Therefore, we see that, up to a multiplicative constant,

[TABLE]

We shall also need the following estimate for solutions of linear SDEs driven by the Brownian martingale $W$.

Lemma 4.4**.**

Let $\alpha_{1}\in L^{\infty}(\mathfrak{M}_{1}),\alpha_{2}\in L^{\infty}(\mathfrak{M}_{2})$ be progressively measurable processes, and $\beta\in L^{\infty}(\mathfrak{M}_{2})$ be a predictable process. Let $\{Y_{t}\}$ be the solution to the SDE

[TABLE]

Suppose that

[TABLE]

where $M>0$ is a constant. Then, for each $\lambda\in\mathcal{P}(\mathbb{S})$ and each $k\in(1/2,\infty)$,

[TABLE]

where $C=C(k,M)>0$ is a constant depending only on $k,M$,

[TABLE]

for any $k\geq 1$ and any progressively measurable process $\varphi(t)$, and

[TABLE]

for a sufficiently large constant $\kappa>0$ depending only on $k,M$ (e.g. $\kappa=8k^{2}(M+1)^{2}$ will suffice). Therefore,

[TABLE]

Remark 4.5*.*

From now on, for the ease of notation, we shall use the same notation $\mathrm{e}_{t}$ to denote $\exp(\kappa\langle W\rangle_{t})$ with possibly different constants $\kappa$ depending only on $k$ and the $L^{\infty}$ norms of coefficients of SDEs.

Proof.

To simplify notation, we shall denote $Y^{m}=|Y|^{m}\,\mathrm{sgn}(Y)$ for any $m>0$. By Itô’s formula,

[TABLE]

Denote $Z=\sup_{0\leq t\leq T}|Y_{t}|\mathrm{e}_{t}^{-1/(2k)}$. Then

[TABLE]

By the Burkholder–Davis–Gundy inequality,

[TABLE]

where $C_{\ast}>0$ is a universal constant. Choosing $\epsilon_{1}=1/4$ and $\epsilon_{2}>0$ sufficiently small gives

[TABLE]

where $C>0$ denotes a constant depending only on $k,M$. Since $\kappa>4M+2kM+4k^{2}M^{2}$, (4.10) follows easily from the above and (4.11) and Young’s inequality. Notice that, in the above inequality, we have used the fact that $\mathbb{E}(Z^{2k})<\infty$ (or alternatively an localization argument together with $|Z|<\infty$ a.s.), which can be shown by an iteration argument similar to the proof of [7, Theorem 3.10]. ∎

We now turn to the derivation of the stochastic maximum principle. Suppose that $\bar{u}\in\mathcal{A}[0,T]$ is a minimizer of (P), and $\bar{x}(\cdot)$ is the corresponding controlled process. Let $\{I_{\epsilon}\}_{\epsilon>0}$ be an arbitrary family of $\mathcal{B}_{\lambda}([0,\infty))$-measurable subsets of $[0,T]$ such that $\lim_{\epsilon\to 0}|I_{\epsilon}|=0$.

Let $S_{1},S_{2}\subseteq[0,\infty)\times\Omega$ be disjoint optional sets such that $\mathfrak{M}_{1}$ is supported on $S_{1}$ and $\mathfrak{M}_{2}$ on $S_{2}$. An example of such $(S_{1},S_{2})$ is $S_{1}=1\{(t,\omega):L_{1}(t,\omega)=1\}$, $S_{2}=1\{(t,\omega):L_{2}(t,\omega)=1\}$, where

[TABLE]

are the Radon–Nikodym derivatives with respect to the optional $\sigma$-field on $[0,T]\times\Omega$. For arbitrary $u_{1},u_{2}\in\mathcal{A}[0,T]$, let

[TABLE]

Let

[TABLE]

Then $E$ is progressively measurable. Notice that if $\mathfrak{M}_{2}(E)=0$, then $m_{k,\lambda}([0,\infty);E)=0$ for all $k\in\mathbb{N}_{+}$.

We denote by $x^{\epsilon}(\cdot)$ the controlled process corresponding to $u^{\epsilon}(\cdot)$, and let

[TABLE]

Define the first-order approximating process $y^{\epsilon}(\cdot)$ by

[TABLE]

and the second-order approximating process $z^{\epsilon}(\cdot)$ by

[TABLE]

where, for any function $\varphi:[0,\infty)\times\mathbb{R}\times\mathbb{U}\to\mathbb{R}$, we denote

[TABLE]

Clearly, $\mathrm{supp}(\delta\varphi)\subseteq I_{\epsilon}$. We shall need the following estimates.

Lemma 4.6**.**

Let $E$ be the progressively measurable set defined by (4.12). Then, for each $k\geq 1$, as $\epsilon\to 0$,

[TABLE]

Proof.

We only present the proof of (4.16) and (4.19), since the proof of (4.17) is similar to that of (4.16), while the proof of (4.18) and (4.20) are similar to that of (4.19). The difference between the proof of (4.16) and (4.19) is that the SDE for $\xi^{\epsilon}-y^{\epsilon}$ involves $\xi^{\epsilon}$ as bias terms $\alpha_{1},\alpha_{2},\beta$ in Lemma 4.4 (see (4.22) below), which requires further estimate. This is also the case for $z^{\epsilon}$ and $\xi^{\epsilon}-y^{\epsilon}-z^{\epsilon}$, and hence their estimates are similar to that of $\xi^{\epsilon}-y^{\epsilon}$.

For any function $\varphi:[0,\infty)\times\mathbb{R}\times\mathbb{U}\to\mathbb{R}$, denote

[TABLE]

By (3.2),

[TABLE]

Let $E_{\epsilon}=E\cap(I_{\epsilon}\times\Omega)$. Then $\mathrm{supp}(\delta\varphi)\subseteq E_{\epsilon},\;\varphi=b_{1},b_{2},\sigma$. By Lemma 4.4,

[TABLE]

where $C>0$ denotes a constant depending only on $k,M$, but might be different at various appearances. We should point out that we explicitly include the term $\mathfrak{M}_{1}(E)$ in the last equation to reflect the fact that \mathbb{E}_{\lambda}\big{[}\big{(}\int_{0}^{T}|\delta b_{1}(t)|dt\big{)}^{2k}\big{]}=0 whenever $\mathfrak{M}_{1}(E)=0$. The appearances of $\mathfrak{M}_{1}(E)$ in other estimates are out of the same purpose. The above estimate completes the proof of (4.16). The proof of (4.17) is similar.

We now turn to the proof of (4.19). By the definition of $\tilde{\varphi}(t)$, we have

[TABLE]

Let $\eta^{\epsilon}=\xi^{\epsilon}-y^{\epsilon}$, and

[TABLE]

Then, by (4.21) and the fact that $\delta\varphi=1_{E_{\epsilon}}$ for $\varphi=b_{1},b_{2},\sigma$, we have

[TABLE]

In order to apply Lemma 4.4, since the desired estimates involving $1_{E_{\epsilon}}(t)\mathrm{O}(1)$ follow directly from definition, we need to estimate \mathbb{E}_{\lambda}\big{[}\big{(}\int_{0}^{T}\chi_{1}(t)\,dt\big{)}^{2k}\big{]}, \mathbb{E}_{\lambda}\big{[}\big{(}\int_{0}^{T}\chi_{1}(t)\,d\langle W\rangle_{t}\big{)}^{2k}\big{]} and \mathbb{E}_{\lambda}\big{[}\big{(}\int_{0}^{T}\chi_{1}(t)^{2}\,d\langle W\rangle_{t}\big{)}^{k}\big{]}, where

[TABLE]

We first estimate \mathbb{E}_{\lambda}\big{[}\big{(}\int_{0}^{T}\chi_{1}(t)\,dt\big{)}^{2k}\big{]}. Notice that, by Lemma 2.3 and (4.16),

[TABLE]

Moreover, for any $p>1$,

[TABLE]

which implies that

[TABLE]

Therefore,

[TABLE]

Next, we estimate \mathbb{E}_{\lambda}\big{[}\big{(}\int_{0}^{T}\chi_{1}(t)\,d\langle W\rangle_{t}\big{)}^{2k}\big{]}. For any $p,q>1$, by (4.16),

[TABLE]

which, in view of the fact that $\mathfrak{T}_{4pq^{\prime}k}(\xi^{\epsilon})=\mathrm{o}(1)$, implies that

[TABLE]

Moreover, by Lemma 2.3 again,

[TABLE]

which, by (4.16), implies that

[TABLE]

Therefore,

[TABLE]

We now estimate \mathbb{E}_{\lambda}\big{[}\big{(}\int_{0}^{T}\chi_{1}(t)^{2}\,d\langle W\rangle_{t}\big{)}^{k}\big{]}. Similarly to the above, for any $p>1$,

[TABLE]

which implies that

[TABLE]

Moreover, for any $p>1$, by Young’s inequality,

[TABLE]

which, together with (4.16), implies that

[TABLE]

Hence,

[TABLE]

With the estimates (4.24)–(4.27), we are now in a position to apply Lemma 4.4 and deduce (4.19). The proof of (4.18) is similar to that of (4.19), except that in the derivation, we need to use both (4.16), (4.17), and (4.19).

The proof of (4.20) is also similar in essence to that of (4.19). By a second order Taylor expansion, it is not difficult to see that, for $\varphi=b_{1},b_{2},\sigma$,

[TABLE]

Therefore,

[TABLE]

for $\varphi=b_{1},b_{2}$, and

[TABLE]

Let $\zeta^{\epsilon}=\xi^{\epsilon}-y^{\epsilon}-z^{\epsilon}$, and

[TABLE]

Then, by substituting (4.28) and (4.29) into the SDE of $\xi^{\epsilon}$, we have

[TABLE]

In view of (4.24) and (4.25), in order to apply Lemma 4.4, it suffices to estimate \mathbb{E}_{\lambda}\big{[}\big{(}\int_{0}^{T}\chi_{2}(t)\,dt\big{)}^{2k}\big{]}, \mathbb{E}_{\lambda}\big{[}\big{(}\int_{0}^{T}\chi_{2}(t)\,d\langle W\rangle_{t}\big{)}^{2k}\big{]} and \mathbb{E}_{\lambda}\big{[}\big{(}\int_{0}^{T}\chi_{2}(t)^{2}\,d\langle W\rangle_{t}\big{)}^{k}\big{]}, which can be done similarly to those of $\chi_{1}(t)$ in the above using the established estimates (4.16), (4.17), and (4.19). ∎

Proof of Theorem 3.4.

Let $E$ be the progressively measurable set defined by (4.12). By definition of $J(\cdot)$, we have

[TABLE]

Notice that we have the following approximations

[TABLE]

The approximation (4.30) follows directly from (4.20). The approximation (4.31) follows from $|(\xi^{\epsilon})^{2}-(y^{\epsilon})^{2}|\leq(|\xi|^{\epsilon}+|y^{\epsilon}|)|\xi^{\epsilon}-y^{\epsilon}|$ together with (4.16), (4.17), and (4.19) in Lemma 4.6. For (4.32), in view of $\mathrm{supp}(\delta\varphi)\subseteq E_{\epsilon}=E\cap(I_{\epsilon}\times\Omega)$ and the boundedness of $\partial_{x}\varphi$, we have

[TABLE]

By $\int_{0}^{T}1_{E_{\epsilon}}(t)|\xi^{\epsilon}(t)|^{2}dt\leq|I_{\epsilon}|\mathrm{e}_{T}\sup_{t\in[0,T]}\xi^{\epsilon}(t)\mathrm{e}_{t}^{-1}$ and Lemma 4.6, it is easily seen that

[TABLE]

which yields the approximation (4.32). Therefore,

[TABLE]

Next, we transform the cost $\mathbb{E}_{\lambda}[\partial_{x}h(\bar{x}(T))(y^{\epsilon}(T)+z^{\epsilon}(T))]$ into a cumulative one. By (3.6),

[TABLE]

Notice that the last integral term \mathbb{E}_{\lambda}\big{(}\int_{0}^{T}\delta(\partial_{x}\sigma)(t)q(t)y^{\epsilon}(t)d\langle W\rangle_{t}\big{)} in the above is also of order \mathfrak{M}_{1}(E)\,\mathrm{o}(|I_{\epsilon}|)+\mathrm{o}\big{(}m_{1,\lambda}(I_{\epsilon};E)\big{)}. To see this, by [7, Theorem 3.5], \mathbb{E}_{\lambda}\big{(}\int_{0}^{T}q(t)^{2}\mathrm{e}_{t}d\langle W\rangle_{t}\big{)} is bounded. Therefore, for any $k\geq 2$, in view of $\mathrm{supp}(\delta(\partial_{x}\sigma))\subseteq E_{\epsilon}$ and Lemma 2.3,

[TABLE]

Hence, the equality (4.34) can be further written as

[TABLE]

Also, we transform \mathbb{E}_{\lambda}[\partial_{x}^{2}h\big{(}\bar{x}(T)\big{)}y^{\epsilon}(T)^{2}] into a cumulative cost. By (3.7),

[TABLE]

Similar to before, it can be shown that the term \mathbb{E}_{\lambda}\big{(}\int_{0}^{T}[2\partial_{x}\sigma(t)P(t)+Q(t)]\delta\sigma(t)y^{\epsilon}(t)\,d\langle W\rangle_{t}\big{)} is of order \mathfrak{M}_{1}(E)\,\mathrm{o}(|I_{\epsilon}|)+\mathrm{o}\big{(}m_{1,\lambda}(I_{\epsilon};E)\big{)}. Therefore,

[TABLE]

Combining (4.33), (4.35), and (4.36), we arrive at

[TABLE]

We now show that the optimality of $\bar{u}$ and (4.37) implies that

[TABLE]

By separability of $\mathbb{U}$ and the continuity of $H_{1},H_{2}$ in $u$, there exist progressively measurable processes $\bar{u}_{1}$ and $\bar{u}_{2}$ such that

[TABLE]

We first set $u_{1}=\bar{u}_{1},\,u_{2}=\bar{u}$. Then $\mathfrak{M}_{2}(E)=0$, and therefore $m_{2,\lambda}(I_{\epsilon};E)=0$. Moreover, (4.37) reduces to

[TABLE]

which clearly implies the first inequality in (4.38).

We now turn to the proof of the second inequality in (4.38). For any $a>0$, let

[TABLE]

Set $u_{1}=\bar{u},\,u_{2}=\bar{u}_{2}1_{E_{a}}+\bar{u}1_{E_{a}^{c}}$. Then $E=E_{a}$ and $\mathfrak{M}_{1}(E)=0$. Therefore, (4.37) reduces to

[TABLE]

By the definition of $E_{a}$ and $u_{2}$, we have

[TABLE]

Therefore,

[TABLE]

which clearly implies

[TABLE]

Therefore, \mathfrak{M}_{2}(E_{a})=\mathbb{E}_{\lambda}\big{(}\int_{0}^{T}1_{E_{a}}(t,\omega)d\langle W\rangle_{t}\big{)}=0 in view of the arbitrariness of $\{I_{\epsilon}\}_{\epsilon>0}$. This completes the proof. ∎

5 An example: linear regulator problem

Let $\lambda\in\mathcal{P}(\mathbb{S})$ with $\lambda\ll\nu$ and $a>0$, and take as the decision space $\mathbb{U}=\mathbb{R}$. We consider the following linear regulator problem, which has wide applications in mathematical finance and engineering (see [3, p. 23] and references therein):

[TABLE]

with

[TABLE]

Suppose that $(\bar{x}(\cdot),\bar{u}(\cdot))$ is an optimal pair of the problem (5.1). The adjoint equations are

[TABLE]

[TABLE]

Clearly, $P(t)=-2,\;Q(t)=0$ is the solution to (5.4).

The Hamiltonians are

[TABLE]

Let $\mathfrak{M}_{1},\mathfrak{M}_{2}$ be the measures on $[0,\infty)\times\Omega$ given by (3.4) and (3.5), and $\mathfrak{M}=\mathfrak{M}_{1}+\mathfrak{M}_{2}$. By Theorem 3.4,

[TABLE]

and

[TABLE]

which implies that

[TABLE]

It follows from the above and (5.3) that

[TABLE]

Therefore, $(\bar{x}(\cdot),\bar{u}(\cdot),p(0))$ is given by the system

[TABLE]

where $p(\cdot)$ is given by (5.5). Note that, compared to BSDEs, the system (5.6) takes the random variable $p(0)$ as a part of its solution so that the additional condition $\bar{x}(0)=1$ is satisfied. Therefore, (5.6) is not a simple SDE or BSDE but a forward–backward type SDE.

We now look for a solution to the form $\bar{x}(t)=\theta(t)p(t)$, where $\theta(t)$ is a process of the form

[TABLE]

By Itô’s formula,

[TABLE]

Comparing the above with (5.6) gives that

[TABLE]

[TABLE]

Therefore, $\xi_{1}(t)=\frac{1}{a}\;\;\mathfrak{M}_{1}\text{-a.e.}$ and $\xi_{2}(t)=2\eta(t)-\theta(t)\;\;\mathfrak{M}_{2}\text{-a.e.}$ Furthermore, $\theta(t)$ is given by the BSDE

[TABLE]

of which a unique solution $(\theta(\cdot),\eta(\cdot))$ exists (cf. [7, Theorem 3.10]).

For the moment, let us assume that $\theta(0)<0$. Then by $\bar{x}(0)=\theta(0)p(0)=1$, we have that $p(0)=1/\theta(0)$ and

[TABLE]

The optimal pair $(\bar{x}(\cdot),\bar{u}(\cdot))$ is given by

[TABLE]

and

[TABLE]

where $(\theta(\cdot),\eta(\cdot))$ and $p(\cdot)$ are given by (5.7) and (5.8).

It remains to show that $\theta(0)<0$. Let

[TABLE]

By Itô’s formula,

[TABLE]

Therefore,

[TABLE]

which implies that $\Phi(t)\theta(t)-\frac{1}{a}\int_{0}^{t}\Phi(r)dr$ is a martingale. Therefore,

[TABLE]

which gives that

[TABLE]

This, together with the fact that $\Phi(t)>0$, shows that $\theta(0)<0$.

Acknowledgements

The author would like to thank the anonymous referees for providing many useful comments and suggestions, which helps improve the quality of the current paper.