Singular control of SPDEs with space-mean dynamics
Nacira Agram, Astrid Hilbert, Bernt {\O}ksendal

TL;DR
This paper develops maximum principles for optimal singular control of SPDEs with space-mean dependence, modeling population growth in random environments, and introduces a reflected BSPDE framework with existence and uniqueness results.
Contribution
It introduces a novel control framework for SPDEs with space-mean dependence, including maximum principles and a new class of reflected BSPDEs with proven well-posedness.
Findings
Derived necessary and sufficient maximum principles for control.
Established existence and uniqueness of the reflected BSPDEs.
Applied the theory to optimal population harvesting models.
Abstract
We consider the problem of optimal singular control of a stochastic partial differential equation (SPDE) with space-mean dependence. Such systems are proposed as models for population growth in a random environment. We obtain sufficient and necessary maximum principles for such control problems. The corresponding adjoint equation is a reflected backward stochastic partial differential equation (BSPDE) with space-mean dependence. We prove existence and uniqueness results for such equations. As an application we study optimal harvesting from a population modelled as an SPDE with space-mean dependence.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Singular control of SPDEs with space-mean dynamics
Nacira AGRAM1 Astrid HILBERT1 and Bernt ØKSENDAL2
(3 May 2019)
Abstract
We consider the problem of optimal singular control of a stochastic partial differential equation (SPDE) with space-mean dependence. Such systems are proposed as models for population growth in a random environment. We obtain sufficient and necessary maximum principles for such control problems. The corresponding adjoint equation is a reflected backward stochastic partial differential equation (BSPDE) with space-mean dependence. We prove existence and uniqueness results for such equations. As an application we study optimal harvesting from a population modelled as an SPDE with space-mean dependence.
MSC(2010):
60H05, 60H15, 93E20, 91G80,91B70.
Keywords:
Stochastic partial differential equations; space-mean dependence; maximum principle; backward stochastic partial differential equations; space-mean reaction diffusion equation; optimal harvesting.
11footnotetext: Department of Mathematics, Linnaeus University (LNU), Sweden.
Emails: [email protected], [email protected]: Department of Mathematics, University of Oslo, Norway.
Email: [email protected]. This research was carried out with support of the Norwegian Research Council, within the research project Challenges in Stochastic Control, Information and Applications (STOCONINF), project number 250768/F20.
1 Introduction
We start by a motivation for the problem that will be studied in this paper:
Consider a problem of optimal harvesting from a fish population in a lake . We assume that the density of the population at time and at the point is modelled by a stochastic reaction-diffusion equation with neighbouring interactions. By this we mean a stochastic partial differential equation of the form
[TABLE]
where is the space-averaging operator
[TABLE]
where denotes Lebesgue volume and
[TABLE]
is the ball of radius in centered at [math],where is a bounded Lipschitz domain in and are given deterministic functions.
In the above is an -dimensional Brownian motion on a filtered probability space . Moreover, , and are given constants and
[TABLE]
is the Laplacian differential operator on .
We may regard as the harvesting effort rate, and as the harvesting efficiency coefficient. The performance functional is assumed to be of the form
[TABLE]
where is the unit price of the fish and is the unit cost of energy used in the harvesting and is a fixed terminal time. Thus represents the expected total net income from the harvesting. The problem is to maximise over all (admissible) harvesting strategies .
** **Remark 1.1
This population growth model, which was first introduced in Agram et al [1], is a generalisation of the classical stochastic reaction-diffusion model, in that we have added the term which represents an average of the neighbouring densities. Thus our model allows for the growth at a point to depend on interactions from the whole vicinity. This space-mean interaction is different from the pointwise interaction represented by the Laplacian.
The problem above turns out to be related to a problem of the following form:
Let be an -measurable -valued random variable. Let
[TABLE]
be a given measurable mapping and a given continuous function. Consider the problem to find an -adapted random fields left-continuous and increasing with respect to , such that
[TABLE]
where is a second order linear partial differential operator. We call the equation (1.3) a reflected stochastic partial differential equation (SPDE) with space-mean dynamics. We will come back to this equation in the last section.
2 The optimization problem
We now give a general formulation of the problem discussed in the Introduction:
Let and let be an open set with boundary Specifically, we assume that the state at time and at the point satisfies
[TABLE]
Here is a -dimensional Brownian motion, defined in a complete filtered probability space The filtration is assumed to be the -augmented filtration generated by .
We denote by the second order partial differential operator acting on given by
[TABLE]
where is a given nonnegative definite matrix with entries for all and for all
Let denote the set of real measurable functions on . For each the functions
[TABLE]
are functionals on , where is the Lebesgue measure on . Here is interpreted in the sense of distribution. Thus is understood as a weak (mild) solution to (2.1), in the sense that
[TABLE]
where is the semigroup associated to the operator . Thus we see that we can in the usual way apply the Itô formula to such SPDEs.
Moreover, the adjoint operator of an operator on is defined by the identity
[TABLE]
where is the inner product in In our case we have
[TABLE]
We interpret as a weak (variational) solution to (2.1), in the sense that for
[TABLE]
where represents the duality product between and , with the Sobolev space of order . In the above equation, we have not written all the arguments of , for simplicity.
We want to maximize the performance functional given by
[TABLE]
over all , where is the set of all adapted processes that are nondecreasing and left continuous with respect to for all , with and such that We call the set of admissible singular controls. Thus we want to find such that
[TABLE]
For each we assume that the functions and are functionals on .
The Hamiltonian is defined by
[TABLE]
where
[TABLE]
and
[TABLE]
We assume that and admit Fréchet derivatives with respect to and
In general, if is Fréchet differentiable, we denote its Fréchet derivative (gradient) at by , and we denote the action of on a function by .
Definition 2.1
We say that the Fréchet derivative of a map has a dual function if
[TABLE]
By Fubini’s theorem, we get
[TABLE]
We associate to the Hamiltonian the following reflected BSPDE
[TABLE]
where we have used the simplified notation
[TABLE]
and similarly with .
2.1 A sufficient maximum principle
We now formulate a sufficient version ( a verification theorem) of the maximum principle for the optimal control of the problem (2.1)-(2.5).
Theorem 2.2** (Sufficient Maximum Principle)**
*Suppose , with corresponding
Suppose the functions and
are concave for each . Moreover, suppose that*
[TABLE]
i.e.,
[TABLE]
Then is an optimal singular control.
Proof. Consider
[TABLE]
where
[TABLE]
[TABLE]
and
[TABLE]
By concavity on together with the identity (2.9)-(2.10), we get
[TABLE]
where .
Applying the Itô formula to , we have
[TABLE]
By the first Green formula (see e.g. Wloka [19], page 258) there exist first order boundary differential operators such that
[TABLE]
where the last integral is the surface integral over . We have that
[TABLE]
for all
Substituting in , yields
[TABLE]
Using the definition of the Hamiltonian , we get
[TABLE]
Summing the above we end up with
[TABLE]
By the maximum condition of (2.12), we have
[TABLE]
2.2 A necessary maximum principle
The concavity conditions in the sufficient maximum principle imposed on the involved coefficients are not always satisfied. Hence, we will derive now a necessary optimality conditions which do not require such an assumptions. We shall first need the following Lemmas:
For , we let** ** denote the set of adapted processes of finite variation with respect to , such that there exists , such that for all
Lemma 2.3
Let and choose . Define the derivative process
[TABLE]
Then satisfies the following singular linear SPDE
[TABLE]
Lemma 2.4
Let and . Put . Then
[TABLE]
Proof. By (2.4) and (2.18), we have
[TABLE]
Using the definition (2.6) of the Hamiltonian, yields
[TABLE]
where we have used the simplified notation
[TABLE]
etc.
Applying the Itô formula to , we get
[TABLE]
Since for , we deduce that
[TABLE]
Therefore, substituting (2.21) and (2.20) into (2.19), we get
[TABLE]
We can now state our necessary maximum principle:
Theorem 2.5** (Necessary Maximum Principle)**
(i) Suppose is optimal, i.e.
[TABLE]
Let be the corresponding solution of (2.1) and (2.11), respectively, and assume that (2.17) holds with . Then
[TABLE]
and
[TABLE]
(ii) Conversely, suppose that there exists such that the corresponding solutions of (2.1) and (2.11), respectively, satisfy
[TABLE]
and
[TABLE]
Then is a directional sub-stationary point for , in the sense that
[TABLE]
Proof. The proof is just a consequence of Lemma 2.4 and Theorem 3 in Øksendal et al [13].
3 Application to Optimal Harvesting
We now return to the problem of optimal harvesting from a fish population in a lake stated in the Introduction. Thus we suppose the density of the population at time and at the point is given by the stochastic reaction-diffusion equation
[TABLE]
where is a constant and, as in (1.1),
[TABLE]
The performance criterion is assumed to be
[TABLE]
where and are given deterministic functions. We can interpret as the harvesting effort at .
Problem 3.1
We want to find such that
In this case the Hamiltonian is
[TABLE]
Recall that for the map given by we know that
[TABLE]
See Example 3.1 in Agram et al [1]. Therefore the adjoint equation is
[TABLE]
The variational inequalities for an optimal control and the associated are:
[TABLE]
We claim that
[TABLE]
Suppose this claim is proved. Then, choosing first and then in the above we obtain that
[TABLE]
In addition we get that
[TABLE]
which implies that always.
Summarising, we have proved the following:
Theorem 3.2
Suppose that and satisfies the following variational inequality
[TABLE]
Then is an optimal singular control for the space-mean SPDE singular control problem (3.1)
We see that this, together with (3.2) constitute a reflected BSPDE, albeit of a slightly different type than the one that will be discussed in the next section.
We summerize the above in the following:
Theorem 3.3
(a)
Suppose is an optimal singular control for the harvesting problem
[TABLE]
where is given by the SPDE (3.1). Then solves the reflected BSPDE (3.2), (3.4).
(b)
Conversely, suppose is a solution of the reflected BSPDE (3.2), (3.4). Then is an optimal control for the problem to maximize the performance (1.2).
Heuristically we can interpret the optimal harvesting strategy as follows:
- •
As long as , we do nothing.
- •
If , we harvest immediately from at a rate which is exactly enough to prevent from dropping below in the next moment.
- •
If , we harvest immediately what is necessary to bring up to the level of
** **Remark 3.4
Note that if and
[TABLE]
then an immediate harvesting of an amount from produces an immediate decrease in the process and hence pushes below This follows from the comparison theorem for reflected BSPDEs of the type (3.2).
4 Existence and uniqueness of solutions of space-mean reflected
backward SPDEs
Let be two separable Hilbert spaces such that is continuously, densely imbedded in . Identifying with its dual we have
[TABLE]
where we have denoted by the topological dual of . Let be a bounded linear operator from to satisfying the following Gårding inequality (coercivity hypothesis): There exist constants and so that
[TABLE]
where denotes the action of on and (respectively ) the norm associated to the Hilbert space (respectively ). We will also use the following spaces:
- •
is the set of all Lebesgue measurable such that
[TABLE]
- •
is the set of -measurable -valued random variables such that .
We let and
Denote by the barrier which is a measurable function that is differentiable in time and twice differentiable in space such that
[TABLE]
is a -valued continuous process, nonnegative, nondecreasing in and
We now consider the adjoint equation (2.11) as a reflected backward stochastic evolution equation
[TABLE]
where stands for the -valued continuous process and the solution of equation is understood as an equation in the dual space of .
We mean by the differential operator with respect to , while is the partial differential operator with respect to , and
[TABLE]
The following result is essential due to Agram et al [1]:
Lemma 4.1
For all we have
[TABLE]
We shall now state and prove our main result of existence and uniqueness of solutions to reflected BSPDE.
Theorem 4.2** (Existence and uniqueness of solutions)**
The space-mean reflected BSPDE has a unique solution -valued progressively measurable process, provided that the following assumptions hold:
(i)
The terminal condition is -measurable random variable and satisfies
[TABLE]
(ii)
There exists a constant such that
[TABLE]
for all
Proof. For the proof of the theorem, we introduce the penalized backward SPDEs:
[TABLE]
According to Agram et al [1], the solution of the above equation (4.4) exists and is unique. We are going to show that forms a Cauchy sequence, i.e.,
[TABLE]
[TABLE]
[TABLE]
Applying Itô’s formula, it follows that
[TABLE]
Now we estimate each of the terms on the right side:
[TABLE]
By the Lipschitz continuity of and the inequality , together with inequality (4.3), one has
[TABLE]
It follows from (4.5) and (4.6) that
[TABLE]
Gronwall inequality, yields
[TABLE]
and
[TABLE]
By inequality (4.7) and the Burkholder inequality we get
[TABLE]
Under the conditions of Theorem 4.2 and by Lemma 5 in Øksendal et al [13], there exists a constant such that
[TABLE]
Denote by , the limit of and , respectively. Put
[TABLE]
Inequality (4.8) implies that admits a non-negative weak limit, denoted by , in the following Hilbert space:
[TABLE]
with inner product
[TABLE]
Set . Then is a continuous -valued process which is increasing in . Letting in (4.4) we obtain
[TABLE]
Inequality (4.8) and the Fatou Lemma imply that . In view of the continuity of in , we conclude a.e. in , for every . Combining the strong convergence of and the weak convergence of , we also have
[TABLE]
Hence,
[TABLE]
We have shown that is a solution to the reflected backward SPDE .
Uniqueness. Let , be two such solutions to equation . By Itô’s formula, we have
[TABLE]
Similar to the proof of existence, we have
[TABLE]
and
[TABLE]
On the other hand,
[TABLE]
Combining (4.11)-(4.14) we arrive at
[TABLE]
Appealing to the Gronwall inequality, this implies
[TABLE]
which further gives from the equation they satisfy.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] Agram, Nacira, Hilbert, Astrid & Øksendal, Bernt: SPD Es with Space-Mean Dynamics. ar Xiv:1807.07303 (2018).
- 2[2] Bensoussan, A. (1983): Maximum principle and dynamic programming approaches of the optimal control of partially observed diffusions. Stochastics 9(3), 169-222.
- 3[3] Bensoussan, A. (1991): Stochastic maximum principle for systems with partial information and application to the separation principle. Applied Stochastic Analysis. Gordon and Breach, 157-172.
- 4[4] Bensoussan, A. (2004): Stochastic Control of Partially Observable Systems. Cambridge University Press.
- 5[5] Donati-Martin, Catherine & Pardoux, Etienne (1993): White noise driven SPD Es with reflection. ,Probability Theory and Related Fields 95(1),1-24.
- 6[6] Holden, H. , Øksendal, B., Ubøe, J. & Zhang, T. (2010): Stochastic Partial Differential Equations. A Modelling, White Noise Functional Approach. Springer Universitext, Second Edition.
- 7[7] Hu, Y., Ma, J., & Yong, J. (2002): On semi-linear degenerate backward stochastic partial differential equations. Probability Theory and Related Fields, 123(3), 381-411.
- 8[8] Hu, Y., & Peng, S. (1990): Maximum principle for semilinear stochastic evolution control systems. Stochastics and Stochastic Reports, 33(3-4), 159-180.
