Singular Optimal Controls for Stochastic Recursive Systems under Convex Control Constraint
Liangquan Zhang
TL;DR
This paper develops second-order necessary conditions and a verification theorem for singular optimal controls in stochastic systems governed by FBSDEs, broadening the theoretical framework and linking maximum principle with dynamic programming.
Contribution
It introduces new second-order necessary conditions and a viscosity solution-based verification theorem for singular controls in stochastic systems, extending existing theories.
Findings
Derived pointwise second-order necessary conditions for stochastic SOCs.
Established a verification theorem for SOCs using viscosity solutions.
Connected maximum principle with dynamic programming without smoothness assumptions.
Abstract
In this paper, we study two kinds of singular optimal controls (SOCs for short) problems where the systems governed by forward-backward stochastic differential equations (FBSDEs for short), in which the control has two components: the regular control, and the singular one. Both drift and diffusion terms may involve the regular control variable. The regular control domain is postulated to be convex. Under certain assumptions, in the framework of the Malliavin calculus, we derive the pointwise second-order necessary conditions for stochastic SOC in the classical sense. This condition is described by two adjoint processes, a maximum condition on the Hamiltonian supported by an illustrative example. A new necessary condition for optimal singular control is obtained as well. Besides, as a by-product, a verification theorem for SOCs is derived via viscosity solutions without involving any…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStochastic processes and financial applications · Markov Chains and Monte Carlo Methods · Risk and Portfolio Optimization
Singular Optimal Controls for Stochastic Recursive Systems under
Convex Control Constraint
Liangquan Zhang1
- School of Science,
Beijing University of Posts and Telecommunications,
Beijing 100876, China L. Zhang acknowledges the financial support partly by the National Nature Science Foundation of China(Grant No. 11701040, 61871058 &11871010) and the Fundamental Research Funds for the Central Universities (No.2019XD-A11). E-mail: [email protected].
Abstract
In this paper, we study two kinds of singular optimal controls (SOCs for short) problems where the systems governed by forward-backward stochastic differential equations (FBSDEs for short), in which the control has two components: the regular control, and the singular one. Both drift and diffusion terms may involve the regular control variable. The regular control domain is postulated to be convex. Under certain assumptions, in the framework of the Malliavin calculus, we derive the pointwise second-order necessary conditions for stochastic SOC in the classical sense. This condition is described by two adjoint processes, a maximum condition on the Hamiltonian supported by an illustrative example. A new necessary condition for optimal singular control is obtained as well. Besides, as a by-product, a verification theorem for SOCs is derived via viscosity solutions without involving any derivatives of the value functions. It is worth pointing out that this theorem has wider applicability than the restrictive classical verification theorems. Finally, we focus on the connection between the maximum principle and the dynamic programming principle for such SOCs problem without the assumption that the value function is smooth enough.
AMS subject classifications: 93E20, 60H15, 60H30.
**Key words: **Dynamic programming principle (DPP for short), Forward-backward stochastic differential equations (FBSDEs for short), Malliavin calculus, Maximum principle (MP for short), Singular optimal controls, Viscosity solution, Verification theorem.
1 Introduction
Singular stochastic control problem is a fundamental topic in fields of stochastic control. This problem was first introduced by Bather and Chernoff [11] in 1967 by considering a simplified model for the control of a spaceship. It was then found that there was a connection between the singular control and optimal stopping problem. This link was established through the derivative of the value function of this initial singular control problem and the value function of the corresponding optimal stopping problem. Subsequently, it was considered by Beněs, Shepp, Witzsenhausen (see [6]) and Karatzas and Shreve (see [53, 54, 55, 56, 57]).
The state process is described by a -dimensional SDE of the following type:
[TABLE]
on some filtered probability space , where are given deterministic functions, is an -dimensional Brownian motion, are initial time and state, is a regular control process, and , with nondecreasing left-continuous with right limits stands for the singular control111Because the measure may be singular with respect to the Lebesgue measure . (SC for short). To avoid the risk of confusion, we shall introduce the other definitions of singular control in various senses. Indeed, they are just a coincidence of terminology usage.
The aim is to minimize the cost functional:
[TABLE]
where
[TABLE]
are given deterministic functions, where represents the running cost tare of the problem and the cost rate of applying the singular control.
We mention that there are four approaches to deal with singular control: The first, partial differential equations (PDE for short) and on variational arguments, can be found in the works of Alvarez [1, 2], Chow, Menaldi, and Robin [24], Karatzas [54], Karatzas and Shreve [57], and Menaldi and Taksar [64]. The second one is related to probabilistic methods; see Baldursson [7], Boetius [8, 9], Boetius and Kohlmann [10], El Karoui and Karatzas [31, 32], Karatzas [53], and Karatzas and Shreve [55, 56]. Third, the DPP, has been studied in a general context, for example, by Boetius [9], Haussmann and Suo [43], Fleming and Soner [33] and Zhang [93]. At last the maximum principle for optimal singular controls (see, for example, Cadenillas and Haussmann [21], Dufour and Miller [28], Dahl and Øksendal [29] see references therein).
Singular controls are used in diverse fields such as mathematical finance (see Baldursson and Karatzas [12], Chiarolla, Haussmann [22], Kobila [58], Karatzas, Wang [59], Davis, Norman [26] and Pagès and Possamaï [76]), manufacturing systems (see, Shreve, Lehoczky, and Gaver [79]), and queuing systems (see Martins and Kushner [65]).
Completely different from the singular control introduced above, to the best of our knowledge, there are two other types of singular optimal controls, in which the first-order necessary conditions turn out to be trivial. We list briefly as follows:
- •
Singular optimal control in the classical sense (SOCCS for short), is the optimal control for which the gradient and the Hessian of the corresponding Hamiltonian with respect to the control variable vanish/degenerate.
- •
Singular optimal control in the sense of Pontryagin-type maximum principle (SOCSPMP for short), is the optimal control for which the corresponding Hamiltonian is equal to a constant in the control region.
When an optimal control is singular in certain senses above (SOCCS and SOCSPMP), usually the first-order necessary condition could not carry sufficient information for the further theoretical analysis and numerical computation, and consequently it is necessary to investigate the second order necessary conditions. In the deterministic setting, reader can refer many articles in this direction (see [5, 34, 39, 40, 50, 51, 52] and references therein).
As for the second-order necessary conditions for stochastic singular optimal controls (SOCCS and SOCSPMP), there are some work should be mentioned, for instance [89, 90] (note that singular control in these articles does not appear in systems). Tang [81] obtained a pointwise second order maximum principle for stochastic singular optimal controls in the sense of the Pontryagin-type maximum principle whenever the control variable does not enter into the diffusion term. Meanwhile, Tang addressed an integral-type second-order necessary condition for stochastic optimal controls with convex control constraints. Zhang and Zhang [89] also establish certain pointwise second-order necessary conditions for stochastic singular (SOCCS) optimal controls, in which both drift and diffusion terms in may depend on the control variable with convex control region by making use of Malliavin calculus technique. Later, adopting the same idea but with large complicated analysis, Zhang et al. [90] deepen this research for the general case when the control region is nonconvex.
The theory of backward stochastic differential equation (BSDE for short) can be traced back to Bismut [3, 4] who studied linear BSDE motivated by stochastic control problems. Pardoux and Peng 1990 [74] proved the well-posedness for nonlinear BSDE. Duffie and Epstein (1992) introduced the notion of recursive utilities in continuous time, which is actually a type of BSDE where the generator is independent of . El Karoui et al. (1997, 2001) extended the recursive utility to the case where contains . The term can be interpreted as an ambiguity aversion term in the market (see Chen and Epstein 2002 [25]). Particularly, the celebrated Black-Scholes formula indeed provided an effective way of representing the option price (which is the solution to a kind of linear BSDE) through the solution to the Black-Scholes equation (parabolic partial differential equation actually). Since then, BSDE has been extensively studied and used in the areas of applied probability and optimal stochastic controls, particularly in financial engineering (cf for instance [48]).
By means of BSDE, Peng (1990) [72] considered the following type of stochastic optimal control problem: Minimize a cost function
[TABLE]
subject to
[TABLE]
over an admissible control domain which need not be convex, and the diffusion coefficients depends on the control variable. In his paper, by spike variational method and the second order adjoint equations, Peng [72] obtained a general stochastic maximum principle for the above optimal control problem. It was just the adjoint equations in stochastic optimal control problems that motivated the famous theory of BSDE (cf [74]).
Later, Peng first [73] studied a stochastic optimal control problem where state variables are described by the system of FBSDEs:
[TABLE]
where and are given deterministic constants. The optimal control problem is to minimize the cost function:
[TABLE]
over an admissible control domain which is convex. Later, Xu [86] studied the following non-fully coupled forward-backward stochastic control system:
[TABLE]
The optimal control problem is to minimize the cost function over but the control domain is non-convex. Wu [84] firstly gave the maximum principle for optimal control problem of fully coupled forward-backward stochastic system:
[TABLE]
where is a random variable and the cost function:
[TABLE]
The optimal control problem is to minimize the cost function over an admissible control domain which is convex. Ji and Zhou [47] obtained a maximum principle for stochastic optimal control of non-fully coupled forward-backward stochastic system with terminal state constraints. Shi and Wu [80] studied the maximum principle for fully coupled forward-backward stochastic system:
[TABLE]
and the cost function is
[TABLE]
The control domain is non-convex but the forward diffusion does not contain the control variable.
Subsequently, in order to study the backward linear-quadratic optimal control problem, Kohlmann and Zhou [60], Lim and Zhou [63] developed a new method for handling this problem. The term is regarded as a control process and the terminal condition as a constraint, and then it is possible to use the Ekeland variational principle to obtain the maximum principle. Adopting this idea, Yong [88] and Wu [85] independently established the maximum principle for the recursive stochastic optimal control problem (noting the diffusion term containing control variable with non-convex control region). Nonetheless, the maximum principle derived by these method involves two unknown parameters. Therefore, the hard questions raise as follows: What is the second-order variational equation for the BSDE? How to obtain the second-order adjoint equation since the quadratic form with respect to the variation of . All of which seem to be extremely complicated.
Hu [44] overcomes the above difficulties by introducing two new adjoint equations. Then, the second-order variational equation for the BSDE and the maximum principle are obtained. The main difference of his variational equations with those in Peng [72] consists in the term in the variation of . Due to the term in the variation of , Hu obtained a global maximum principle which is novel and different from that in Wu [85], Yong [88] and previous work, which solves completely Peng’s open problem. Furthermore, Hu’s maximum principle is stronger than the one in Wu [85], Yong [88]. For a general case, reader can refer [45].
Motivated by above work, in this paper, we consider singular controls problem of the following type:
[TABLE]
with the similar cost functional
[TABLE]
Wang [83] firstly introduced and studied a class of singular control problems with recursive utility, where the cost function is determined by BSDE. Under certain assumptions, the author proved that the value function is a nonnegative, convex solution of the H-J-B equation. However, FBSDEs in Wang [83] do not contain the regular control and the generator is not general case. In our work, using some properties of the BSDE and analysis technique, we expand the extension of the MP for SOC to the recursive control problem in Zhang and Zhang [89]. To the best of our knowledge, such singular optimal controls problems of FBSDEs (8) via two kinds of singular controls have not been explored before. We shall establish some pointwise second-order necessary conditions for stochastic optimal controls of FBSDEs. Both drift and diffusion terms may contain the control variable , and we assume that the control region is convex. We also consider the pointwise second-order necessary condition, which is easier to verify in practical applications.
As claimed in [89], quite different from the deterministic setting, there exist some essential difficulties in deriving the pointwise second-order necessary condition from an integral-type one whenever the diffusion term depends on the control variable, even for the case of convex control domain. We overcome these difficulties by means of some technique from the Malliavin calculus. For general case, namely, the control region is non-convex can be found in [90].
In this paper, we are interested in studying singular optimal controls for FBSDEs (8). Compared with above literature, our paper has several new features. The novelty of the formulation and the contribution in this paper may be stated as follows:
- •
Our control systems in this paper are governed by FBSDEs which exactly extends the work of Zhang and Zhang [89] to utilities. Our work is the first time to establish the pointwise second order necessary condition for stochastic singular optimal control in the classical sense for FBSDEs, a new necessary condition for singular control is involved as well. In this sense, our paper actually considers two kinds of singular controls problems simultaneously, which is interesting to deepen this research.
- •
We derive a new verification theorem for optimal singular controls via viscosity solution, which responses to the question raised in Zhang [93]; Meanwhile, we study the relationship between the adjoint equations derived and value function, which extends the smooth case considered by Cadenillas and Haussmann [21] to the framework of viscosity solution for stochastic recursive systems.
The rest of this paper is organized as follows: after some preliminaries in the second section, we are devoted the third section to the MP for two kinds of singular optimal controls. A concrete example is concluded with as well. Then, in Section 4, we study the verification theorem for singular optimal controls via viscosity solutions. Finally, we establish the relationship between the DPP and MP for viscosity solution. Some proofs of lemmas are displayed in Appendix 5.
2 Preliminaries and Notations
Throughout this paper, we denote by the space of -dimensional Euclidean space, by the space the matrices with order . Let be a complete filtered probability space on which a one-dimensional standard Brownian motion is defined, with being its natural filtration, augmented by all the -null sets. Given a subset (nonempty, bounded, and convex) of we will denote , separately, the class of measurable, adapted processes with nondecreasing left-continuous with right limits and , moreover, is called singular control. For each , we denote by the natural filtration of the Brownian motion , augmented by the -null sets of . appearing as superscript denotes the transpose of a matrix. In what follows, represents a generic constant, which can be different from line to line.
We now introduce the following spaces of processes:
[TABLE]
and denote Clearly, forms a Banach space.
For any we study the stochastic control systems governed by FBSDEs (8).
We assume that the following conditions hold:
(A1)
The coefficients are twice continuously differentiable with respect to are continues in are bounded , are bounded by for some positive constant Moreover, for any
[TABLE]
(A2)
The coefficients are twice continuously differentiable with respect to is a given deterministic matrix. are continuous in . There exists constant such that for any
[TABLE]
[TABLE]
[TABLE]
[TABLE]
and
[TABLE]
Under above assumptions (A1)-(A2), for any , it is easy to check that FBSDEs (8) admit a unique -adapted solution denoted by the triple (See Pardoux and Peng [74]).
Like Peng [75], given any control processes , we introduce the following cost functional:
[TABLE]
We are interested in the value function of the stochastic optimal control problem:
[TABLE]
Since the value function (11) is defined by the solution of controlled FBSDEs (8), so from the existence and uniqueness, is well-defined.
The following estimate is very useful whose proof can be found in Briand et al. 2003 [19].
Lemma 1**.**
Let be the solution to the following
[TABLE]
where and whilst satisfies the conditions (A2), and
[TABLE]
Then, for some there exists a positive constant such that
[TABLE]
Particularly, whenever putting and one has
[TABLE]
Now let us recall briefly the notion of differentiation on Wiener space (see the expository papers by Nualart 1995 [66], Nualart and Pardoux [67] and Ocone 1988 [69]).
- •
will denote the set of functions of class from into whose partial derivatives of order less than or equal to are bounded.
- •
Let denote the set of random variables of the form where , and .
- •
If is of the above form, we define its derivative as being the -dimensional process
[TABLE]
For we define the norm
[TABLE]
It can be shown (Nualart 1995) that the operator has a closed extension to the space , the closure of with respect to the norm . Observe that if is -measurable, then for . We denote by , the ith component of .
Let denote the set of -valued progressively measurable processes such that
- •
For a.e.
- •
admits a progressively measurable version;
- •
We have
[TABLE]
Note that for each is an matrix. Hence, Obviously, is defined uniquely up to sets of measure zero. Moreover, denote by the set of all adapted processes in
We define the following notations from Zhang and Zhang [89]:
[TABLE]
and
[TABLE]
Denote For any denote Whenever is adapted, it follows that for Furthermore, since Put as the set of all adapted processes in
3 Maximum Principle of Singular Optimal Controls
This section will study the optimal controls separately. Due to the special structure of control systems, we shall first consider the singular control part, deriving the necessary condition, subsequently, regular part. The initial condition will fixed to be At the beginning let us suppose that is an optimal control and denote by the optimal solution of (8). Our maximum principle will be proved in two steps. The first variational inequality is derived from the fact
[TABLE]
where is a convex perturbation of optimal control. The second variational inequity is attained from the inequity
[TABLE]
where is a convex perturbation of
3.1 Optimal Singular Control
For we denote
[TABLE]
Let us introduce the following
Proposition 2**.**
Let (A1)-(A2) hold, and let be an optimal solution. Then, the following FBSDEs:
[TABLE]
admit an adapted solution
Theorem 3**.**
Let (A1)-(A2) hold. If is an optimal solution of (8), then there exists a unique pair of adapted processes satisfying (15) such that
[TABLE]
and
[TABLE]
Before the proof, we need some lemmas. At the beginning, we introduce the convex perturbation
[TABLE]
where and is an arbitrary element of We now introduce the following variational equations of (8):
[TABLE]
From (A1)-(A2) it is easy to check that (18) has a unique strong solution. Moreover, we have
Lemma 4**.**
Under the Assumptions (A1)-(A2), we have
[TABLE]
The proof can be seen in the Appendix.
Proof of Theorem 3.
Applying Itô’s formula to on yields
[TABLE]
In particular, let be a process satisfying and such that (22) and
[TABLE]
holds where denotes the th component. Then,
[TABLE]
Thus
[TABLE]
which proves (17). Next we show that (16) is valid. For that, let us define the events:
[TABLE]
where
Define the stochastic process by
[TABLE]
Then one can easily check that is a measurable, adapted process which is nondecreasing left-continuous with right limits and , and which satisfies
[TABLE]
Further, we have
[TABLE]
which obviously contradicts to (22), unless for any we have We thus complete the proof.
Remark 5**.**
One can easily check that
[TABLE]
which implies that , -a.s. So makes sense. Clearly, our Theorem 3 for optimal singular control is completely different from [13]. Ours contains two variables . As a matter of fact, we have
[TABLE]
We claim that is the partial derivative of value function, which will be studied in Section 4.3.
3.2 Optimal Regular Control
In this subsection, we study the optimal regular controls for systems driven by FBSDEs (8) under the types of Pontryagin, namely, necessary maximum principles for optimal control. To this end, we fix and introduce the following convex perturbation control. Taking we define where is sufficiently small. Since is convex, Let be the trajectory of the control system (8) corresponding to the control Put When and we denote
[TABLE]
Let us introduce the following two kinds of variational equations, mainly taken from [17]. For simplicity, we omit the superscript.
[TABLE]
and
[TABLE]
From Lemma 3.5 and Lemma 3.11 in [17], we have following result.
Lemma 6**.**
Assume that (A1)-(A2) is in force. Then, we have, for any
[TABLE]
where
[TABLE]
We shall introduce the so called variational equations for FBSDEs (8) beginning from the following two adjoint equations:
[TABLE]
and
[TABLE]
where are unknown two processes to be determined. Next we will derive two kinds of adjoint equations. The main idea is borrowed from [44]. First of all, we observe that
[TABLE]
which inspires us to use the adjoint equations to expand the following:
[TABLE]
Itô’s formula applied to (27) yields for
[TABLE]
where
[TABLE]
Remark 7**.**
Note that and do not appear in the -term.
Define
[TABLE]
Let
[TABLE]
Clearly, from Lemma 6, we have
[TABLE]
After some tedious computations, we have
[TABLE]
Put
[TABLE]
then we attain
[TABLE]
where
[TABLE]
Next we are going to seek determined by the optimal quadruple such that
[TABLE]
where
[TABLE]
in which does not involve the terms and Note that in BSDE (30), there appears the term Hence, we make use of Taylor’s expansion to
[TABLE]
where the Hessian matrix is with respect to
Then, we obtain
[TABLE]
where denotes the identity matrix.
In the classical theory of optimal control for FBSDEs (cf [84, 85]), there generally appear two groups of the first-order adjoint equations, for instance in Eqs. (15). The following proposition will establish the relationship between them with from (25), which is very useful to study the connection between maximum principle and dynamic programming (see Theorem 40 below).
Proposition 8**.**
Suppose that Assumptions (A1)-(A2) are in force. Then we have
[TABLE]
where and are solutions to FBSDEs (25) and (15), respectively.
The proof is just to apply the Itô’s formula to so we omit it.
We define the classical Hamiltonian function:
[TABLE]
where
Then, we have
[TABLE]
Namely,
[TABLE]
Remark 9**.**
Note that FBSDEs (33) are somewhat different from (22) in Hu [44]. Specifically, the term disappears in (22) since
[TABLE]
in [44] by using spike variational approach. Nevertheless, the corresponding term in our paper is just . We will see a moment later that this term is needed to define an extensive “Hamiltonian function” as follows.
Define
[TABLE]
where
[TABLE]
We now give the adjoint equation for BSDE (33) as follows:
[TABLE]
Lemma 10**.**
Under the Assumptions (A1)-(A2), SDE (34) admits a unique adapted strong solution Moreover, we have
[TABLE]
Proof.
The first inequality can be obtained from Theorem 6.16 of [87]. We deal with the second one. By Itô’s formula, we have
[TABLE]
It follows that
[TABLE]
But by the B-D-G inequality, we get
[TABLE]
Thus
[TABLE]
The second estimation comes from the Hölder inequality. We complete the proof.
Set
[TABLE]
We are able to give the variational equations as follows:
[TABLE]
and
[TABLE]
Obviously, we have
[TABLE]
Lemma 11**.**
Under the Assumptions (A1)-(A2), we have the following estimation
[TABLE]
Proof.
To prove (37), we consider (33) again. From assumptions (A1)-(A2), one can check that the adjoint equations (25) and (26) have a unique adapted strong solution, respectively. Furthermore, by classical approach, we are able to get the following estimates for
[TABLE]
Applying Lemma 1 to (33), we get the desired result. Indeed, since there appears a term
[TABLE]
in BSDE (33), so we have the estimation with We complete the proof.
We shall derive a variational inequality which is crucial to establish the necessary condition for optimal control. Before this, we introduce the following the other type of singular control using the Hamiltonian function:
Definition 12** (Singular control in the classical sense).**
We call a control a singular control in the classical sense if satisfies
[TABLE]
*where denotes the state trajectories driven by Moreover, and denote the adjoint processes given respectively by (25) and (26) with replaced by
. If this is also optimal, then we call it a singular optimal control in the classical sense.*
Remark 13**.**
Hu [44] first considers the forward-backward stochastic control problem whenever the diffusion term depends on the control variable with non-convex control domain. In order to to establish the stochastic maximum principle, he introduces the -function of the following type:
[TABLE]
Note that this Hamiltonian function is slightly different from Peng 1990 [72]. The main difference of this variational equations with those in (Peng 1990) [72] appears in the term (the similar term in our paper) in variational equation for BSDE and maximum principle for the definition of in the variation of , which is for any order expansion of . So it is not helpful to use the second-order Taylor expansion for treating this term. The stochastic maximum principle (see [44]) says that if is an optimal pair, then
[TABLE]
Apparently, Definition 12 says that a singular control in the classical sense is the real one that fulfils trivially the first and second-order necessary conditions in classical optimization theory dealing with the maximization problem (40), namely,
[TABLE]
It is easy to verify that (39) is equivalent to (41). Certainly, one could investigate stochastic singular optimal controls for forward-backward stochastic systems in other senses, say, in the sense of process in Skorohod space, which can be seen in Zhang [93] via viscosity solution approach (Hamilton-Jacobi-Bellman inequality), or in the sense of Pontryagin-type maximum principle (cf Tang [81]). As this complete remake of the various topics is much longer than the present paper, it will be reported elsewhere.
Lemma 14** (Variational inequality).**
Under the Assumptions (A1)-(A2), it holds that
[TABLE]
where
[TABLE]
Proof.
Using Itô’s formula to on we get the desired result.
Theorem 15**.**
Assume that (A1)-(A2) hold. If is a singular optimal control in the classical sense, then
[TABLE]
for any
Proof.
According to the definition of value function, we have
[TABLE]
Letting we get the desired result from Definition 12 and Lemma 14.
Remark 16**.**
Clearly, if does not depend on , then Consequently, (43) reduces to
[TABLE]
which is just the classical case studied in Zhang et al. [89] for classical stochastic control problems. Meanwhile, our result actually extends Peng [73] to second order case.
Remark 17**.**
Recall that, for deterministic system, it is possible to derive pointwise necessary conditions for optimal controls via the first suitable integral-type necessary conditions and normally there is no obstacles to establish the pointwise first-order necessary condition for optimal controls whenever an integral type one is on the hand. Nevertheless, the classical approach to handle the pointwise condition from the integral-type can not be employed directly in the framework of the pointwise second-order condition in the general stochastic setting because of certain feature the stochastic systems owning. In order to derive the second order variational equations for BSDE in Hu [44], the author there introduces two kinds of adjoint equations and a new Hamiltonian function. The main difference of this variational equations with those in (Peng 1990) [72] lies in the term Then, it is possible to get the maximum principle basing one variational equation. Note that the order of the difference between perturbed state, optimal state and first, second order state is
As observed in Theorem 15, there appears a term . In order to deal with it, we give the expression of mainly taken from Theorem 1.6.14 in Yong and Zhou [87]. To this end, consider the following matrix-valued stochastic differential equation:
[TABLE]
where denotes the identity matrix in . Then,
[TABLE]
Substituting the explicit representation (45) of into (43) yields
[TABLE]
Clearly, (46) contains an Itô’s integral. Next we shall borrow the spike variation method from [89] to check its order with perturbed control. More precisely, let and be a Borel set with Borel measure define
[TABLE]
where This is called a spike variation of the optimal control . For our aim, we only need to use for and . Let
[TABLE]
Then, inserting it into (46), we have
[TABLE]
By Hölder inequality and Burkholder-Davis-Gundy inequality, we have
[TABLE]
since from classical estimate for stochastic differential equations.
Lemma 18** (Martingale representation theorem).**
Suppose that Then, there exists a such that
[TABLE]
The proof can seen in Zhang et al. [89].
Lemma 19**.**
Assume that (A1)-(A2) hold. Then,
[TABLE]
Proof.
We shall prove that
[TABLE]
From (A1)-(A2), we have
[TABLE]
Besides,
[TABLE]
Hence,
[TABLE]
From Lemma 10 and the classical estimation in (38), we finish the proof.
Therefore, by our assumption (A1)-(A2) and Lemma 18, for any , there exists a
[TABLE]
such that for a.e.
[TABLE]
Using (47), we are able to assert the following:
Theorem 20**.**
Suppose that (A1)-(A2) are in force. Let be a singular optimal control in the classical sense, then we have
[TABLE]
where
[TABLE]
where is obtained by (47), and is determined by (44).
The proof is just to repeat the process in Theorem 3.10, [89], so we omit it.
Note that Theorem 20 is pointwise with respect to the time variable (but also the integral form). Now if each of and are regular enough, then the function admits an explicit representation.
Suppose the following:
(A3)
Theorem 21**.**
Suppose that the Assumptions (A1)-(A3) are in force. Let be a singular optimal control in the classical sense, then we have
[TABLE]
Observe that the expression (43) is similar to (3.17) in [89]. Therefore, the proof is repeated as in Theorem 3.13 in Zhang and Zhang [89].
3.2.1 Example
We provide a concrete example to illustrate our theoretical result (Theorem 21) by looking at an example. If the FBSDEs considered in this paper are linear, it is possible to implement our principles directly. For convenience, we still adopt the notations introduced in Section 3.2.
Example 22**.**
Consider the following FBSDEs with and
[TABLE]
One can easily get the solutions to (34),
[TABLE]
Set The corresponding adjoint equations are (25) and (26), namely,
[TABLE]
and
[TABLE]
We get immediately, the solutions to (49) and (50) are
[TABLE]
respectively. Hence, we have
[TABLE]
Therefore, is a singular control in the classical sense. Moreover, we compute
[TABLE]
Consequently, we get
[TABLE]
which indicates that Theorem 21 always holds and is a singular optimal control.
4 Singular Optimal Controls via Dynamic Programming Principle
In this section, we proceed our control problem from the view point of DPP. From now on, we focus on the following
[TABLE]
Since the value function defined by the solution of controlled BSDE (51), so from the existence and uniqueness, defined in (11) is well-defined.
Remark 23**.**
We assume that and are deterministic matrices. On the one hand, from the derivations in Theorem 5.1 of [43], it is convenient to show the “inaction” region for singular control; On the other hand, we may regard together as a solution, in this way, we are able to apply the classical Itô’s formula, avoiding the appearance of jump. We believe these assumptions can be removed properly, but at present, we consider constant only in our paper. Whilst in order to get the uniqueness of the solution to H-J-B inequality (52), we add the assumption More details, see Theorem 2.2 in [93].
Set
[TABLE]
4.1 Verification Theorem via Viscosity Solutions
Zhang [93] has given a verification theorem for smooth solution of the following H-J-B inequality:
[TABLE]
Lemma 24**.**
Define
[TABLE]
Then the optimal state process is continuous whenever . To be precise, we have
[TABLE]
The proof can be seen in Zhang [93].
Proposition 25**.**
Suppose that is a classical solution of the H-J-B inequality (52) such that for some
[TABLE]
Then for any ,
[TABLE]
Furthermore, if there exists such that
[TABLE]
and
[TABLE]
Then
[TABLE]
In this section, we remove the unreal condition, smooth on value function, by means of viscosity solutions222In the classical optimal stochastic control theory, the value function is a solution to the corresponding H-J-B equation whenever it has sufficient regularity (Fleming and Rishel [35], Krylov [49]). Nevertheless, when it is only known that the value function is continuous, then, the value function is a solution to the H-J-B equation in the viscosity sense (see Lions [23]).. We will recall the definition of a viscosity solution for H-J-B variational inequality (52) from [23]. Below, will denote the set of symmetric matrices.
Let us begin at introducing the following parabolic superjet:
Definition 26**.**
Let and . We denote by , the “parabolic superjet” of at the set of triples which are such that
[TABLE]
Similarly, we denote by the “parabolic subjet” of at the set of triples which are such that
[TABLE]
Lemma 27**.**
Let and be given. Then:
1) if and only if there exists a function such that attains a strict maximum at and
[TABLE]
2) if and only if there exists a function such that attains a strict minimum at and
[TABLE]
More details can be seen in Lemma 5.4 and 5.5 in Yong and Zhou [87].
Define
[TABLE]
Definition 28**.**
(i) It can be said is a viscosity subsolution of (52) if , and at any point , for any ,
[TABLE]
In other words, at any point we have both and
[TABLE]
(ii) It can be said is a viscosity supersolution of (52) if , and at any point , for any ,
[TABLE]
In other words, at any point where , we have
[TABLE]
(iii) It can be said is a viscosity solution of (52) if it is both a viscosity sub and super solution.
We have the following result:
Proposition 29**.**
Assume that (A1)-(A2) are in force. Then there exists at most one viscosity solution of H-J-B inequality (52) in the class of bounded and continuous functions.
We need a generalized Itô’s formula. Define
[TABLE]
For any , by virtue of Doléans-Dade-Meyer formula (see [43, 21]), we have
[TABLE]
We begin to introduce a useful lemma.
Lemma 30**.**
Assume that (A1)-(A2) are in force. Let be fixed and let be an admissible pair. Define processes
[TABLE]
Then
[TABLE]
The proof can be found in [87].
Lemma 31**.**
Let Extend to with for and for Suppose that there is a integrable function and some such that
[TABLE]
Then
[TABLE]
The proof can be seen in Zhang [92].
The main result in this section is the following.
Theorem 32** (Verification Theorem).**
Suppose that the Assumptions (A1)-(A2) are in force. Let be a viscosity solution of the H-J-B equations (52), satisfying the following conditions:
[TABLE]
Then we have
[TABLE]
for any and any
Furthermore, let be fixed and let
[TABLE]
be an admissible pair such that there exist a function and a triple
[TABLE]
satisfying
[TABLE]
and
[TABLE]
where and is defined in (58). Then is an optimal pair.
In order to prove Theorem 32, we need the following lemma:
Lemma 33**.**
Let be a viscosity subsolution of the H-J-B equations (52) satisfying (63). Then we have
[TABLE]
where
The proof can be seen in the Appendix.
Proof of Theorem 32.
We have (64) from the uniqueness of viscosity solutions of the H-J-B equations (52). It remains to show that is an optimal, we now fix such that (65) and (66) hold at For We claim that the set of such points is of full measure in by Lemma 7 in [92]. Now we fix such that the regular conditional probability , given is well defined. In this new probability space, the random variables are almost surely deterministic constants and equal to
[TABLE]
respectively. We remark that in this probability space the Brownian motion is still the a standard Brownian motion although now almost surely. The space is now equipped with a new filtration and the control process is adapted to this new filtration. For -a.s. the process is a solution of (1.1) on in with the inial condition
Then on the probability space , we are going to apply Itô’s formula to on for any
[TABLE]
Taking conditional expectation value dividing both sides by , and using (66), we have
[TABLE]
We now handle the last two terms. Note that
[TABLE]
and
[TABLE]
Thus
[TABLE]
We now deal the term
[TABLE]
Combining (70) and (71), we have
[TABLE]
Letting and employing the similar delicate method as in the proof of Theorem 4.1 of Gozzi et al. [41], we have
[TABLE]
From Lemma 33, that there exist and such that
[TABLE]
and
[TABLE]
holds, respectively.
By virtue of Fatou’s Lemma, noting (74), we obtain
[TABLE]
for a.e. Then the rest of the proof goes exactly as in [41].
We apply Lemma 8 in [92] to using (73), 67) and (75) to get
[TABLE]
From this we claim that
[TABLE]
where
[TABLE]
Thus, combining the above with the first assertion (64), we prove the is an optimal pair. The proof is thus completed.
Remark 34**.**
The condition (67) is just equivalent to the following:
[TABLE]
where is defined in Theorem 32. This is easily seen by recalling the fact that is the viscosity solution of (52):
[TABLE]
Remark 35**.**
Clearly, Theorem 32 is expressed in terms of parabolic superjet. One could naturally ask whether a similar result holds for parabolic subjet. The answer was positive for the deterministic case (in terms of the first-order parabolic subjet, see Theorem 3.9 in [87]). Unfortunately, as claimed in Yong and Zhou [87], the answer is that the statement of Theorem 32 is no longer valid whenever the parabolic superjet in (66) is replaced by the parabolic subjet.
Now let us present a non-smooth version of the necessity part of Theorem 32. However, we just have “partial” result.
Theorem 36**.**
Assume that (A1)-(A2) hold. Let be a viscosity solution of the H-J-B equations (52) and let be an optimal singular controls. Let be an admissible pair such that there exist a function and a triple
[TABLE]
satisfying
[TABLE]
Then, it holds that
[TABLE]
Proof.
On the one hand, let and such that
[TABLE]
By Lemma 27, we have a test function with such that achieves its minimum at and
[TABLE]
holds. Then for sufficiently small , a.e. .
[TABLE]
The last inequality comes from the derivation in Theorem 32 by means of the condition (77). On the other hand, since is optimal, by DPP of optimality, it yields
[TABLE]
which implies that
[TABLE]
Therefore, it follows from (78) that
[TABLE]
where
[TABLE]
We thus complete the proof.
4.2 Optimal Feedback Controls
In this subsection, we describe the method to construct optimal feedback controls by the verification Theorem 32. First, let us recall the definition of admissible feedback controls.
Definition 37**.**
A measurable function from to is called an admissible feedback control pair if for any there is a weak solution of the following SDE:
[TABLE]
where is an -valued -adapted right continuous and left limit martingale vanishing in which is orthogonal to the driving Brownian motion Here is the smallest filtration and generated by , which is such that is -adapted. Obviously, is a part of the solution of BSDE of (79). Simultaneously, we suppose that satisfies the Lipschitz condition with respect to . An admissible feedback control pair is called optimal if
[TABLE]
is optimal for each is a solution of (79) corresponding to
Theorem 38**.**
Let be an admissible feedback control and and be measurable functions satisfying for all If
[TABLE]
and for all then is singular optimal control pair.
Proof
From Theorem 32, we get the desired result.
Remark 39**.**
In FBSDEs (79), is actually determined by Hence, we need to investigate the conditions imposed in Theorem 32 to ensure the existence and uniqueness of in law and the measurability of the multifunctions to obtain that minimizes (80). This can be done by virtue of the celebrated Filippov’s Lemma (cf [87]).
4.3 The Connection between DPP and MP
In Section 3, we have obtained the first and second order adjoint equations. In this part, we shall investigate the connection between the general DPP and the MP for such singular controls problem without the assumption that the value is sufficient smooth. By associated adjoint equations and delicate estimates, it is possible to establish the set inclusions among the super- and sub-jets of the value function and the first-order and second- order adjoint processes as well as the generalized Hamiltonian function.
Theorem 40**.**
Assume that (A1)-(A2) are in force. Suppose that be a singular optimal controls, is a value function, and is optimal trajectory. Let and be the adjoint equations (25), (26), respectively. Then, we have
[TABLE]
Proof.
From Theorem 3 and Proposition 8, we get the first part of (81). From Theorem 3.1 in Nie, Shi and Wu [68], we get the second and third results of (81).
5 Concluding remarks
In this paper, on the one hand, we have derived a second order pointwise necessary condition for singular optimal control in classical sense of FBSDEs with convex control domain by means of the variation equations and two adjoint equations, which is separately extends the work by Zhang and Zhang [89] to stochastic recursive case, and Hu [44] to pointwise case in the framework of Malliavin calculus. A new necessary condition for singular control has been obtained. Moreover, we investigate the verification theorem for optimal controls via viscosity solution and establish the connection between the adjoint equations and value function also in viscosity solution sense.
There are still several interesting topics should be scheduled as follows:
- •
As an important issue, the existence of optimal singular controls has never been exploited. Haussmannand and Suo [42] apply the compactification method to study the classical and singular control problem of Itô’s type of stochastic differential equation, where the problem is reformulated as a martingale problem on an appropriate canonical space after the relaxed form of the classical control is introduced. Under some mild continuity assumptions on the data, they obtain the existence of optimal control by purely probabilistic arguments. Note that, in the framework of BSDE with singular control, the trajectory of seems to be a càdlàg process (from French, for right continuous with left hand limits). Hence, we may consider in some space with appropriate topologies, for instance, Skorokhod topology or Meyer-Zheng topology (see [36]) to obtain the convergence of probability measures deduced by involving relaxed control. Related work from the technique of PDEs can be seen in [14, 18] references therein. From Wang [83], one may construct the optimal control via the existence of diffusion with refections (see [24]). However, it is interesting to extend this result to FBSDEs.
- •
The matrices are deterministic. It is also interesting to extend this restriction to time varying matrices, even the generator involving the singular control. Whenever the coefficients are random, the H-J-B inequality will become stochastic PDEs. No doubt, stochastic viscosity solution will be applied. For this direction, reader can refer to Buckdahn, Ma [15, 16], Peng [71] and Qiu [77].
- •
As for the general cases, i.e., the control regions are assumed to be non-convex and both the drift and diffusion terms depend on the control variable. Indeed, such a mathematical model, from view point of application, is more reasonable and urgent in many real-life problems (for instance, some finance models in which the controls may impact the uncertainty, etc). In near future, we shall remove the condition of convex control region, employing the idea developed by Zhang et al. [90]. It is worth mentioning that the analysis in [90] is much more complicated. Some new and useful tools, such as the multilinear function valued stochastic processes, the BSDE for these processes are introduced. Hence, it will be interesting to borrow these tools to investigate the singular optimal controls problems for FBSDEs, which will definitely promote and enrich the theories of FBSDEs.
Appendix A Proofs of Lemmas
Proof of Lemma 4.
We first prove the continuity of solution depending on parameter.
Set
[TABLE]
It can be shown that
[TABLE]
by standard estimates and the Burkholder-Davis-Gundy inequality, so we omit it.
Next, set
[TABLE]
Note that (19) has be obtained in [13]. We will prove (20) and (21).
Then,
[TABLE]
where
[TABLE]
Simple calculation yields
[TABLE]
where
[TABLE]
and
[TABLE]
From classical theory of BSDE, one can show
[TABLE]
By using (82) and (84), the dominated convergence theorem, Lemma 1 and Gronwall’s lemma, we get the desired result by letting .
Proof of Lemma 33.
From (63) and (6) in Gozzi et al. in [41], we have that if then
[TABLE]
We shall deal with separately. For we have by classical estimate and the assumption For from (7) in [41] and Hölder inequality, we have
[TABLE]
since and the fact
Finally,
[TABLE]
By Itô isometry and classical estimate on SDE, we complete the proof.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] L. Alvarez, A class of solvable singular stochastic control problems, Stochastics Stochastics Rep . 67 1999 83–122.
- 2[2] L. Alvarez, Singular stochastic control, linear diffusions, and optimal stopping: A class of solvable problems, SIAM J. Control Optim. 39 2001 1697–1710.
- 3[3] J. M. Bismut, Théorie Probabiliste du Contrôle des Diffusions, Memoirs of the American Mathematical Society , 176, Providence, Rhode Island, 1973.
- 4[4] J. Bismut, An introductory approach to duality in optimal stochastic control, SIAM Riew , 20 1978, 62–78.
- 5[5] D. J. Bell and D. H. Jacobson, Singular Optimal Control Problems, Math. Sci. Eng. 117, Academic Press, New York, 1975.
- 6[6] V.E. Beněs, L.A. Shepp, H.S. Witzsenhausen, Some solvable stochastic control problems, Stochastics 4 1980 39–83.
- 7[7] F. Baldursson, Singular stochastic control and optimal stopping, Stochastics 21 1987 1-40.
- 8[8] F. Boetius, Bounded variation singular stochastic control and associated Dynkin game, in Mathematical Finance, Trends Math., M. Kohlmann and S. Tang, eds., Birkhäuser, Basel, 2001 111–120.