Deterministic Control of Stochastic Reaction-Diffusion Equations

TL;DR

This paper develops a method for controlling stochastic reaction-diffusion equations using deterministic controls, deriving optimality conditions, and applying a conjugate gradient method to approximate solutions, with applications to the Schl"ogl model.

Contribution

It introduces a novel approach to control stochastic PDEs with deterministic controls, avoiding backward SPDEs, and provides a practical gradient-based optimization method.

Findings

Derived necessary conditions for optimal controls with multiplicative noise.

Presented a gradient approximation method for stochastic control problems.

Applied the approach to the stochastic Schl"ogl model.

Abstract

We consider the control of semilinear stochastic partial differential equations (SPDEs) via deterministic controls. In the case of multiplicative noise, existence of optimal controls and necessary conditions for optimality are derived. In the case of additive noise, we obtain a representation for the gradient of the cost functional via adjoint calculus. The restriction to deterministic controls and additive noise avoids the necessity of introducing a backward SPDE. Based on this novel representation, we present a probabilistic nonlinear conjugate gradient descent method to approximate the optimal control, and apply our results to the stochastic Schl\"ogl model. We also present some analysis in the case where the optimal control for the stochastic system differs from the optimal control for the deterministic system.

Equations

$$\{\begin{array}{ll}\mathrm{d}{u}_t^g=\left[\Delta u_t^g+f\left(u_t^g\right)+b(t)g(t)\right]\mathrm{d}t+\sigma (t,u_t^g)\mathrm{d}{W}_t^Q& \text{on }{L}^2(\Lambda )\\ u_0^g(x)=u^0(x)& x\in \Lambda \end{array}$$

$$\begin{array}{rl}J(u^g,g):=& \mathbb{E}\left[\frac{c_{\overline{\Lambda }}}{2}{\int }_0^T{\int }_{\Lambda }{\left(u_t^g(x)-u_{\overline{\Lambda }}(t,x)\right)}^2\mathrm{d}x\mathrm{d}t\right]\\ & +\mathbb{E}\left[\frac{c_T}{2}{\int }_{\Lambda }{\left(u_T^g(x)-u^T(x)\right)}^2\mathrm{d}x\right]+\frac{\lambda }{2}{\int }_0^T{\int }_{\Lambda }{g}^2(t,x)\mathrm{d}x\mathrm{d}t,\end{array}$$

$${\Vert (\sigma (t,u)-\sigma (t,v))\circ \sqrt{Q}\Vert }_{\text{HS}(L^2(\Lambda ))}^2$$

$${\Vert \sigma (t,u)\circ \sqrt{Q}\Vert }_{\text{HS}(L^2(\Lambda ))}^2$$

$${\Vert {\sigma }^{\prime }(t,u)v\circ \sqrt{Q}\Vert }_{\text{HS}(L^2(\Lambda ))}^2$$

$$\underset{x\in \mathbb{R}}{\sup }{f}^{\prime }(x)<\infty ,$$

$$|f^{\prime }(x)|<C(1+|x{|}^2),$$

$$(f(u)-f(v))(u-v)\le {\widetilde{\text{Lip}}}_f(u-v{)}^2,$$

$$\sigma (u):=\bar{\sigma }\min \{u(u-1),M\},$$

$$H^1(\Lambda )\subset L^6(\Lambda )\subset {\left(H^1(\Lambda )\right)}^{\ast },$$

$$E:=L^2([0,T]\times \Omega ,\mathrm{d}t\otimes \mathbb{P};H^1(\Lambda ))\cap L^6(\Omega ,\mathbb{P};C([0,T];L^2(\Lambda )))$$

$$I_1(v):=\mathbb{E}\left[\frac{c_{\overline{\Lambda }}}{2}{\int }_0^T{\int }_{\Lambda }{\left(v(t,x)-u_{\overline{\Lambda }}(t,x)\right)}^2\mathrm{d}x\mathrm{d}t+\frac{c_T}{2}{\int }_{\Lambda }{\left(v(T,x)-u^T(x)\right)}^2\mathrm{d}x\right]$$

$$I_2(g):=\frac{\lambda }{2}{\int }_0^T{\int }_{\Lambda }{g}^2(t,x)\mathrm{d}x\mathrm{d}t,$$

$$J(g):=I_1(u^g)+I_2(g),$$

$$g\in G_{\text{ad}}:=\left\{g\in L^6\left([0,T]\times \Lambda \right)|\Vert g{\Vert }_{L^6([0,T]\times \Lambda )}\le \kappa \right\},$$

$${\tilde{G}}_{\text{ad}}:=\left\{g\in L^{\infty }\left([0,T]\times \Lambda \right)|g_a\le g(t,x)\le g_b\text{ for a.a. }(t,x)\in [0,T]\times \Lambda \right\},$$

$$\mathbb{E}\left[\underset{t\in [0,T]}{\sup }{\Vert u_t^g\Vert }_{L^2(\Lambda )}^6+{\left({\int }_0^T{\Vert u_t^g\Vert }_{H^1(\Lambda )}^2\mathrm{d}t\right)}^3\right]\le C\left(1+{\int }_0^T{\Vert g(t)\Vert }_{L^2(\Lambda )}^6\mathrm{d}t\right).$$

$$\begin{array}{rl}& {\Vert u_t^g\Vert }_{L^2(\Lambda )}^2\\ =& \Vert u^0{\Vert }_{L^2(\Lambda )}^2+2{\int }_0^t{}_{(H^1(\Lambda ){)}^{\ast }}{⟨\Delta u_s^g,u_s^g⟩}_{H^1(\Lambda )}\mathrm{d}s+2{\int }_0^t{⟨f\left(u_s^g\right),u_s^g⟩}_{L^2(\Lambda )}\mathrm{d}s\\ & +2{\int }_0^t{⟨b(s)g(s),u_s^g⟩}_{L^2(\Lambda )}\mathrm{d}s+{\int }_0^t\Vert \sigma (s,u_s^g)\circ \sqrt{Q}{\Vert }_{\text{HS}(L^2(\Lambda ))}^2\mathrm{d}s\\ & +2{\int }_0^t{⟨u_s^g,\sigma (s,u_s^g)d{W}_s^Q⟩}_{L^2(\Lambda )}\\ \le & \Vert u^0{\Vert }_{L^2(\Lambda )}^2-2{\int }_0^t{\Vert \nabla u_s^g\Vert }_{L^2(\Lambda )}^2\mathrm{d}s+\left(2{\widetilde{\text{Lip}}}_f+C+\Vert b{\Vert }_{L^{\infty }([0,T]\times \Lambda )}\right){\int }_0^t{\Vert u_s^g\Vert }_{L^2(\Lambda )}^2\mathrm{d}s\\ & +\Vert b{\Vert }_{L^{\infty }([0,T]\times \Lambda )}{\int }_0^t{\Vert g(s)\Vert }_{L^2(\Lambda )}^2\mathrm{d}s+T|\Lambda |C+2\left|{\int }_0^t{⟨u_s^g,\sigma (s,u_s^g)d{W}_s^Q⟩}_{L^2(\Lambda )}\right|,\end{array}$$

$${\int }_0^t{⟨f(u_s^g),u_s^g⟩}_{L^2(\Lambda )}\mathrm{d}s\le {\widetilde{\text{Lip}}}_f{\int }_0^t{\Vert u_s^g\Vert }_{L^2(\Lambda )}^2\mathrm{d}s.$$

$$\begin{split}\mathbb{E}\left[\sup_{t\in[0,T]}\left\|u^{g}_{t}\right\|_{L^{2}(\Lambda)}^{6}\right]\leq C\Bigg{(}&1+\int_{0}^{T}\mathbb{E}\left[\sup_{s\in[0,t]}\left\|u^{g}_{s}\right\|^{6}_{L^{2}(\Lambda)}\right]\mathrm{d}t+\int_{0}^{T}\left\|g(t)\right\|^{6}_{L^{2}(\Lambda)}\mathrm{d}t\\ &+\mathbb{E}\left[\sup_{t\in[0,T]}\left|\int_{0}^{t}\left\langle u^{g}_{s},\sigma(s,u^{g}_{s})dW^{Q}_{s}\right\rangle_{L^{2}(\Lambda)}\right|^{3}\right]\Bigg{)}.\end{split}$$

$$\begin{array}{r}\mathbb{E}\left[\underset{t\in [0,T]}{\sup }{\left|{\int }_0^t{⟨u_s^g,\sigma (s,u_s^g)\mathrm{d}{W}_s^Q⟩}_{L^2(\Lambda )}\right|}^3\right]\\ \le C\mathbb{E}\left[{⟨{\int }_0^{\cdot }⟨u_s^g,\sigma (s,u_s^g)\mathrm{d}{W}_s^Q{⟩}_{L^2(\Lambda )}⟩}_T^{\frac{3}{2}}\right].\end{array}$$

$$\begin{array}{rl}{⟨{\int }_0^{\cdot }⟨u_s^g,\sigma (s,u_s^g)\mathrm{d}{W}_s^Q{⟩}_{L^2(\Lambda )}⟩}_T& ={\int }_0^T\sum _{k=1}^{\infty }|⟨u_s^g,(\sigma (s,u_s^g)\circ \sqrt{Q})e_k{⟩}_{L^2(\Lambda )}{|}^2\mathrm{d}s\\ & \le {\int }_0^T\sum _{k=1}^{\infty }\Vert u_s^g{\Vert }_{L^2(\Lambda )}^2\Vert (\sigma (s,u_s^g)\circ \sqrt{Q})e_k{\Vert }_{L^2(\Lambda )}^2\mathrm{d}s\\ & ={\int }_0^T\Vert \sigma (s,u_s^g)\circ \sqrt{Q}{\Vert }_{\text{HS}(L^2(\Lambda ))}^2\Vert u_s^g{\Vert }_{L^2(\Lambda )}^2\mathrm{d}s.\end{array}$$

$$\begin{array}{rl}\mathbb{E}\left[\underset{t\in [0,T]}{\sup }{\left|{\int }_0^t{⟨u_s^g,\sigma (s,u_s^g)\mathrm{d}{W}_s^Q⟩}_{L^2(\Lambda )}\right|}^3\right]& \le C\mathbb{E}\left[{\left({\int }_0^T\Vert u_s^g{\Vert }_{L^2(\Lambda )}^2+\Vert u_s^g{\Vert }_{L^2(\Lambda )}^4\mathrm{d}s\right)}^{\frac{3}{2}}\right]\\ & \le C\left(1+{\int }_0^T\mathbb{E}\left[\underset{s\in [0,t]}{\sup }\Vert u_s^g{\Vert }_{L^2(\Lambda )}^6\right]\mathrm{d}t\right)\end{array}$$

$$\mathbb{E}\left[\underset{t\in [0,T]}{\sup }{\Vert u_t^g\Vert }_{L^2(\Lambda )}^6\right]\le C\left(1+{\int }_0^T{\Vert g(t)\Vert }_{L^2(\Lambda )}^6\mathrm{d}t\right).$$

$$\begin{array}{rl}& \mathbb{E}\left[{\left({\int }_0^T{\Vert \nabla u_t^g\Vert }_{L^2(\Lambda )}^2\mathrm{d}t\right)}^3\right]\\ \le & C\left(1+\mathbb{E}\left[{\int }_0^T{\Vert u_t^g\Vert }_{L^2(\Lambda )}^6\mathrm{d}t\right]+{\int }_0^T{\Vert g(t)\Vert }_{L^2(\Lambda )}^6\mathrm{d}t\right).\end{array}$$

$$\mathbb{E}\left[{\left({\int }_0^T{\Vert \nabla u_t^g\Vert }_{L^2(\Lambda )}^2\mathrm{d}t\right)}^3\right]\le C\left(1+{\int }_0^T{\Vert g(t)\Vert }_{L^2(\Lambda )}^6\mathrm{d}t\right).$$

$${\Vert u_t^g\Vert }_{L^6(\Lambda )}^6\le C{\Vert u_t^g\Vert }_{H^1(\Lambda )}^2{\Vert u_t^g\Vert }_{L^2(\Lambda )}^4.$$

$$\begin{array}{rl}\mathbb{E}\left[{\int }_0^T{\Vert u_t^g\Vert }_{L^6(\Lambda )}^6\mathrm{d}t\right]& \le \mathbb{E}\left[{\int }_0^T{\Vert u_t^g\Vert }_{H^1(\Lambda )}^2{\Vert u_t^g\Vert }_{L^2(\Lambda )}^4\mathrm{d}t\right]\\ & \le \mathbb{E}\left[\underset{t\in [0,T]}{\sup }{\Vert u_t^g\Vert }_{L^2(\Lambda )}^4{\int }_0^T{\Vert u_t^g\Vert }_{H^1(\Lambda )}^2\mathrm{d}t\right]\\ & \le \mathbb{E}\left[\underset{t\in [0,T]}{\sup }{\Vert u_t^g\Vert }_{L^2(\Lambda )}^6\right]\mathbb{E}\left[{\left({\int }_0^T{\Vert u_t^g\Vert }_{H^1(\Lambda )}^2\mathrm{d}t\right)}^3\right]\\ & <\infty ,\end{array}$$

$$L^2\left([0,T]\times \Lambda \right)$$

$$g$$

$${\Vert u_t^{g_1}-u_t^{g_2}\Vert }_E^2\le C{\int }_0^T{\Vert g_1-g_2\Vert }_{L^2(\Lambda )}^2\mathrm{d}s$$

$$\begin{array}{rl}{\Vert u_t^{g_1}-u_t^{g_2}\Vert }_{L^2(\Lambda )}^2=2& {\int }_0^t{}_{(H^1(\Lambda ){)}^{\ast }}{⟨\Delta \left(u^{g_1}-u^{g_2}\right),u^{g_1}-u^{g_2}⟩}_{H^1(\Lambda )}\mathrm{d}s\\ & +2{\int }_0^t{⟨f\left(u^{g_1}\right)-f\left(u^{g_2}\right),u^{g_1}-u^{g_2}⟩}_{L^2(\Lambda )}\mathrm{d}s\\ & +2{\int }_0^t{⟨b\left(g_1-g_2\right),u^{g_1}-u^{g_2}⟩}_{L^2(\Lambda )}\mathrm{d}s\\ & +{\int }_0^t\Vert (\sigma (s,u_s^{g_1})-\sigma (s,u_s^{g_2}))\circ \sqrt{Q}{\Vert }_{\text{HS}(L^2(\Lambda ))}^2\mathrm{d}s\\ & +2{\int }_0^t⟨u_s^{g_1}-u_s^{g_2},(\sigma (s,u_s^{g_1})-\sigma (s,u_s^{g_2}))\mathrm{d}{W}_s^Q{⟩}_{L^2(\Lambda )}.\end{array}$$

Full text

\unmarkedfntext

Mathematics Subject Classification (2020) — Primary: 93E20, 49K45; Secondary: 49M05, 60H15, 35K57.

\unmarkedfntext

Keywords and phrases — Stochastic reaction-diffusion equations, variational approach, optimal control, stochastic Schlögl model, stochastic Nagumo equation, nonlinear conjugate gradient descent.

\unmarkedfntext

Email: •  stannat@math.tu-berlin.de  •  wessels@math.tu-berlin.de

1. 1 Introduction

2. 2 General Setting and Well-Posedness of the Optimal Control Problem

3. 3 First Order Condition for Critical Points

4. 4 The Gradient of the Cost Functional

5. 5 Nonlinear Conjugate Gradient Descent

6. 6 Application to Optimal Control of the Stochastic Schlögl Model

7. 6.1 Steering of a Wave Front

8. 6.2 Comparison with the Control of the Deterministic System

9. 6.3 Mathematical Analysis in a Simplified Setting

1 Introduction

In this paper our objective is to investigate the optimal control of the semilinear SPDE

[TABLE]

on bounded domains $\Lambda\subset\mathbb{R}$ with $f$ satisfying a one-sided Lipschitz condition. Here, the control $g$ is deterministic. Precise assumptions on the coefficients of (1) will be stated at the beginning of the following section. In the case where $f(u)=ku(1-u)(u-a)$ for $k>0$ and $a\in(0,1)$, equation (1) is called the stochastic Schlögl model.

We will be interested in the optimal control of (1) w.r.t. the following quadratic cost functional

[TABLE]

where $u_{\overline{\Lambda}}:[0,T]\times\Lambda\to\mathbb{R}$ and $u^{T}:\Lambda\to\mathbb{R}$ are a given running and terminal reference profile, respectively, and $c_{\overline{\Lambda}},c_{T},\lambda\geq 0$ are constants. The optimal control of the deterministic counterpart of (1) (i.e. $\sigma\equiv 0$) has been well studied in the existing literature (see the monograph [25]). In particular, the optimal control of the deterministic Schlögl model has been studied in a series of papers by Tröltzsch, Ryll et al. ([2], [23], [24]).

Recent years have seen a rising interest in the optimal control of SPDEs. Whereas there exists already a quite substantial literature on the dynamic programming approach to the optimal control of SPDEs (see, e.g., the monographs [5] and [10], and in particular [4] and [20] for the case of stochastic reaction-diffusion systems) direct variational methods have been much less applied.

Results concerning existence of optimal controls of nonlinear SPDEs have first been obtained in [16] in the case of the stochastic Navier-Stokes equation, see also the recent work [7] for a discussion of existence of optimal controls for the FitzHugh-Nagumo system with additive noise. Necessary first order conditions for optimality are discussed by Fuhrman et al. in [12, 13] within the mild approach to SPDEs. However, in [12] the authors work under the assumption that the Gâteaux derivative of the nonlinearity is bounded (cf. Assumption 2.1), which does not include the Schlögl model; in [13] the nonlinearity does cover the Schlögl model, but the noise is only additive in this work. Since both these works formulate the problem in the setting of mild solutions to SPDEs, it does not seem tractable to generalize those results to a polynomial nonlinearity while considering a multiplicative noise term. In [8] the authors derive first order necessary conditions using a rescaling method which exploits a certain structure of the state equation. The problem of sufficient conditions for optimal controls has been investigated in [21]. In this paper, the author derives a sufficient maximum principle for a class of quasilinear SPDEs with a one-dimensional noise term.

In the present paper we will be interested in the optimal control of (1) within the variational approach to SPDEs. With a view towards the efficient numerical approximation we will restrict to deterministic controls, and from Section 4 on also to additive noise. The restriction to additive noise allows us to avoid the backward SPDE for the adjoint state and to obtain a conceptually much simpler representation in terms of a backward random PDE, instead. However, using this adjoint state for numerical approximations would lead to non-adapted controls. A further restriction to deterministic controls allows us to derive a deterministic gradient in terms of the simpler adjoint state (cf. Theorem 4.1). This representation of the gradient gives rise to more efficient numerical approximations of (trivially adapted, since deterministic) optimal open-loop controls. We illustrate our approach in the case of the stochastic Schlögl model.

The paper is organized as follows. In Section 2 we state precise assumptions for our analysis, show the well-posedness of the optimal control problem, and prove the existence of an optimal control. In Section 3 we prove the Gâteaux differentiability of the solution map and the cost functional with respect to the control, and derive a necessary condition for a control to be locally optimal. In Section 4 we restrict ourselves to the case of additive noise. In this setting, we derive a representation for the gradient of the cost functional as well as an equation for the adjoint state that is later on used in the numerical approximation of locally optimal solutions. Furthermore, we deduce the Stochastic Minimum Principle from the necessary conditions from the previous section. In Section 5 we present a probabilistic gradient descent method for the approximation of an optimal control. In Section 6 we apply our results to two examples of the stochastic Schlögl model. In the first example, we show how to accelerate traveling waves and change their direction of travel (cf. Subsection 6.1). The second example is one situation, where the optimal control for the stochastic equation apparently differs from the optimal control for the deterministic counterpart (cf. Subsection 6.2). Since we are not able to give a rigorous proof in this case, we also consider in Subsection 6.3 the simplified setting of a stochastic ordinary differential equation, where one can rigorously prove that the optimal control for the stochastic case and its deterministic counterpart are actually different.

2 General Setting and Well-Posedness of the Optimal Control Problem

Consider the stochastic partial differential equation (1) with homogeneous Neumann boundary conditions, where $\Lambda\subset\mathbb{R}$ is a bounded domain, $T>0$ is fixed, $(W_{t}^{Q})_{t\in[0,T]}$ is a $Q$-Wiener process for some nonnegative, symmetric trace class operator $Q:L^{2}(\Lambda)\to L^{2}(\Lambda)$ on a given filtered probability space $(\Omega,\mathcal{F},\left(\mathcal{F}_{t})_{t\in[0,T]},\mathbb{P}\right)$, $u^{0}\in L^{6}(\Omega,\mathcal{F}_{0},\mathbb{P};L^{2}(\Lambda))$, $b\in L^{\infty}([0,T]\times\Lambda)$, $\sigma:[0,T]\times L^{2}(\Lambda)\to L(L^{2}(\Lambda))$ is Fréchet differentiable for every fixed $t\in[0,T]$ and satisfies for all $t\in[0,T]$ and $u,v\in L^{2}(\Lambda)$

[TABLE]

for some constant $C\in\mathbb{R}$, where $\|\cdot\|_{\text{HS}(L^{2}(\Lambda))}$ denotes the Hilbert-Schmidt norm on the space of all Hilbert-Schmidt operators on $L^{2}(\Lambda)$. Furthermore, $f:\mathbb{R}\to\mathbb{R}$ is continuously differentiable satisfying $f(0)=0$,

[TABLE]

and for all $x\in\mathbb{R}$

[TABLE]

for some constant $C\in\mathbb{R}$.

Remark 2.1**.**

Notice that the upper bound of the derivative implies a one-sided Lipschitz condition, i.e. there exists a constant $\widetilde{\text{Lip}}_{f}\in\mathbb{R}$ such that

[TABLE]

for all $u,v\in\mathbb{R}$. 2. 2.

The nonlinearity in the Schlögl equation satisfies these conditions since the leading coefficient of the polynomial is negative and the derivative is a polynomial of degree 2. 3. 3.

A possible choice for the diffusion coefficient would be the Nemytskii operator associated with $\sigma:\mathbb{R}\to\mathbb{R}$,

[TABLE]

for some constants $\bar{\sigma},M>0$. In the stochastic Schlögl model, this choice imposes noise in particular on the wave front of the resulting traveling wave. For a more detailed discussion, see [18], Example 10.2.

Considering the Gelfand triple

[TABLE]

under appropriate assumptions on the control $g$ (to be specified later), the existence of a variational solution to equation (1) in the space

[TABLE]

is assured (see e.g. [17], Example 5.1.8).

Our objective is to study the optimal control problem associated with the state equation (1). Let $I_{1}:L^{6}(\Omega;C([0,T];L^{2}(\Lambda)))\to\mathbb{R}$ be given by

[TABLE]

and $I_{2}:L^{2}([0,T]\times\Lambda)\to\mathbb{R}$

[TABLE]

where $c_{\overline{\Lambda}}$, $c_{T}$, $\lambda\geq 0$, $u_{\overline{\Lambda}}\in L^{2}\left([0,T]\times\Lambda\right)$, and $u^{T}\in L^{2}(\Lambda)$. We want to minimize the cost functional

[TABLE]

subject to the state equation (1), where

[TABLE]

for given $\kappa\geq 0$.

Remark 2.2**.**

The proof of the Gâteaux differentiability of $g\mapsto u^{g}$ (see Proposition 3.1 below), requires a moment bound of the solution in $L^{6}(\Omega\times[0,T]\times\Lambda)$ due to the upper bound (5) on the derivative $f^{\prime}$ of the nonlinearity. Therefore the minimal requirement for an admissible control is $g\in L^{6}([0,T]\times\Lambda)$.

In the work by Buchholz et al. ([2]) on the deterministic case, the set of admissible controls

[TABLE]

for some $g_{a}<g_{b}$ is considered. We could use the same set in our analysis as well.

Throughout the whole paper, we are going to work under the aforementioned conditions. First we want to show that the control problem is well-posed. In order to do so, we need the following a priori bound for solutions of the state equation (1).

Lemma 2.1**.**

There is a constant $C=C(b,f,\sigma,T,Q,u^{0})$ such that for every solution $u^{g}\in E$ of the state equation (1) associated with $g\in G_{\text{ad}}$ on the right hand side we have

[TABLE]

Proof.

By Itô’s formula (see [17], Theorem 4.2.5), we have

[TABLE]

where we used the growth bound (3b) on $\sigma$ and that by the one-sided Lipschitz continuity of $f$ and $f(0)=0$, we have

[TABLE]

Taking both sides of equation (LABEL:ito) to the power $3$, taking the supremum with respect to $t\in[0,T]$, and taking expectations yields

[TABLE]

By Burkholder-Davis-Gundy inequality (see e.g. [15]), we get

[TABLE]

Now, we compute the quadratic variation. To this end, let $(e_{k})_{k\geq 1}$ be an orthonormal basis of $L^{2}(\Lambda)$. Then

[TABLE]

Using the linear growth condition (3b) we obtain together with equation (19)

[TABLE]

Together with equation (18) and Grönwall’s inequality, this yields

[TABLE]

Furthermore, from (LABEL:ito), we get

[TABLE]

Putting together equations (22) and (23), we get for some constant

$C=C(b,f,\sigma,T,Q,u^{0})$

[TABLE]

Together with (22), this completes the proof. ∎

As a consequence, the finiteness of all of the integrals appearing in the cost functional $J$ is assured. Furthermore, we get the following corollary.

Corollary 2.1**.**

Let $E$ be defined as in (9). Every solution $u^{g}\in E$ of the state equation (1) associated with $g\in G_{ad}$ on the right hand side is in $L^{6}(\Omega\times[0,T]\times\Lambda)$.

Proof.

We apply the Gagliardo-Nirenberg interpolation inequality which can be found in [22]. This yields for almost all $(t,\omega)\in[0,T]\times\Omega$

[TABLE]

Integrating over $[0,T]\times\Omega$ yields

[TABLE]

where we used Hölder’s inequality and Lemma 2.1. ∎

Next, we show that the solution map of the state equation (1) is globally Lipschitz continuous.

Proposition 2.1**.**

Let $E$ be defined as in (9). For the solution map

[TABLE]

there exists a constant $C=C(f,b,\sigma,Q,\Lambda,T)\in\mathbb{R}$ such that

[TABLE]

In particular, the solution map is Lipschitz continuous from $L^{2}\left([0,T]\times\Lambda\right)$ to $E$.

Proof.

By Itô’s formula (see [17], Theorem 4.2.5), we have almost surely

[TABLE]

Using the Lipschitz condition (3a) and similar arguments as in the proof of Lemma 2.1 yields the claim. ∎

Now we want to prove the existence of an optimal control:

Theorem 2.1**.**

There is at least one optimal solution $g^{\ast}\in G_{\text{ad}}$ such that

[TABLE]

Proof.

First, we notice that $J$ is nonnegative and hence bounded from below. Let $\left(g_{n}\right)_{n\in\mathbb{N}}\subset G_{\text{ad}}$ be a minimizing sequence, i.e.

[TABLE]

and let $u^{g_{n}}\in E$ denote the unique solution of the state equation (1) associated with $g_{n}$ on the right hand side.

Since $\left(g_{n}\right)_{n\in\mathbb{N}}\subset G_{\text{ad}}$, $\left(g_{n}\right)_{n\in\mathbb{N}}$ is in particular bounded in $L^{2}\left([0,T]\times\Lambda\right)$. Hence, we can extract a weakly convergent subsequence - again denoted by $g_{n}$ - such that $g_{n}\rightharpoonup g^{\ast}$ in $L^{2}([0,T]\times\Lambda)$. The point is now to show that $g^{\ast}\in G_{\text{ad}}$, and $g^{\ast}$ minimizes $J$ in $G_{\text{ad}}$.

Since $G_{\text{ad}}$ is convex and strongly closed, it follows that $G_{\text{ad}}$ is also weakly closed, hence $g^{\ast}\in G_{\text{ad}}$.

In order to show that $g^{\ast}$ minimizes $J$, we first show that $u^{g_{n}}$ converges strongly to $u^{g^{\ast}}$. In the deterministic case, the a priori bound in Lemma 2.1 holds pathwise and we can apply a compact embedding theorem in order to show strong convergence of the solutions. Since we only have the a priori bound under the expectation, we cannot use the same technique. Instead we apply the so called compactness method introduced in [11]. Let us sketch this technique here:

From the bound

[TABLE]

we can conclude tightness of the measures $\mathbb{P}^{n}:=\mathbb{P}\circ(u^{g_{n}})^{-1}$ on $L^{2}([0,T]\times\Lambda)$. Therefore, $(\mathbb{P}^{n})_{n\in\mathbb{N}}$ is relatively compact and we can extract a converging subsequence $\mathbb{P}^{n}\to\mathbb{P}^{\ast}$. It remains to identify the limit $\mathbb{P}^{\ast}$. By the Skorohod embedding theorem there exists a probability space $(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})$ and a sequence of random variables $(\tilde{u}^{g_{n}})_{n\in\mathbb{N}}$ and $\tilde{u}^{g^{\ast}}$ defined on $\tilde{\Omega}$ with the same law as $(u^{g_{n}})_{n\in\mathbb{N}}$ and $u^{g^{\ast}}$, respectively, such that $\tilde{u}^{g_{n}}\to\tilde{u}^{g^{\ast}}$ strongly in $L^{2}([0,T]\times\Lambda)$ $\tilde{\mathbb{P}}$-almost surely. Therefore, using the martingale representation theorem, we can identify $\tilde{u}^{g^{\ast}}$ as a solution to our state equation associated with $g^{\ast}$ on the right hand side (see [9], Section 8.4 for details).

Now, we split the cost functional into one part that depends on $u^{g}$ and into one part that depends on $g$. For the first part, $I_{1}$, we have

[TABLE]

where we used Fatou’s Lemma, and exploited that uniqueness in law holds for the state equation (1) and we have a solution in the space $E$ (see equation (9)).

Furthermore, since $I_{2}$ is continuous and convex, it is also weakly lower semicontinuous, i.e.

[TABLE]

Therefore, we have

[TABLE]

which completes the proof. ∎

Remark 2.3**.**

This proof does not rely on the explicit form of our cost functional. The crucial point is, that the cost functional is sequentially weakly lower semicontinuous.

3 First Order Condition for Critical Points

In this section, we are first going to derive the Gâteaux derivative of the solution map and the cost functional and then prove a necessary condition for a control to be locally optimal.

Proposition 3.1**.**

Let $f:\mathbb{R}\to\mathbb{R}$ satisfy the assumptions of Section 2 and $g\in L^{6}([0,T]\times\Lambda)$ be fixed. Then, for every $h\in L^{6}([0,T]\times\Lambda)$, the Gâteaux derivative of the solution map $g\mapsto u^{g}$, $L^{6}([0,T]\times\Lambda)\to E$ in direction $h$ is given by the solution of the linear SPDE

[TABLE]

Proof.

The idea for this proof is taken from [19], Theorem 4.4; see also [1], where the same technique is applied in a similar setting. Let $y^{h}$ denote the solution of equation (37) associated with $h$ on the right hand side. Set

[TABLE]

We want to show that $z_{\delta}\to 0$ in $L^{2}(\Omega\times[0,T];H^{1}(\Lambda))\cap L^{2}(\Omega;C([0,T];L^{2}(\Lambda)))$ as $\delta\to 0$. First notice

[TABLE]

Note that

[TABLE]

and similarly

[TABLE]

Together with equation (39), Itô’s formula (see [17], Theorem 4.2.5) yields

[TABLE]

First notice that

[TABLE]

Furthermore, we have $\langle R_{\delta}(s),z_{\delta}(s)\rangle_{L^{2}(\Lambda)}\leq(\|R_{\delta}(s)\|_{L^{2}(\Lambda)}^{2}+\|z_{\delta}(s)\|_{L^{2}(\Lambda)}^{2})/2$, and, since $f$ is one-sided Lipschitz continuous, we have

[TABLE]

For the last term in equation (42), we have

[TABLE]

where

[TABLE]

and, by the Lipschitz condition (3a) on $\sigma$,

[TABLE]

Therefore, taking the supremum with respect to $t\in[0,T]$ in equation (42) and taking expectations, it follows

[TABLE]

Using Burkholder-Davis-Gundy inequality, we have

[TABLE]

Now we compute the quadratic variation. To this end, let $(e_{k})_{k\geq 1}$ be an orthonormal basis of $L^{2}(\Lambda)$. Then we have

[TABLE]

for arbitrary $\varepsilon>0$. With the same estimates as above for $\|\Sigma_{\delta}(s)\circ\sqrt{Q}\|_{\text{HS}(L^{2}(\Lambda))}^{2}$ and with inequality (49) this yields

[TABLE]

Furthermore, with similar calculations as above, we get

[TABLE]

for arbitrary $\varepsilon>0$. Choosing $\varepsilon>0$ in (51) and (52) small enough, we get from (48)

[TABLE]

By Grönwall’s inequality, this yields

[TABLE]

Since $R_{\delta}\to 0$ as $\delta\to 0$ for almost all $(\omega,t,x)\in\Omega\times[0,T]\times\Lambda$, we get by the dominated convergence theorem

[TABLE]

Here, we used that $R_{\delta}$ is dominated in the following way: By assumption (5), Taylor’s formula and elementary estimates, we have

[TABLE]

The boundedness of the right hand side in $L^{2}(\Omega\times[0,T]\times\Lambda)$ follows immediately from Corollary 2.1 (notice that we get the boundedness of $y^{h}$ in $L^{6}(\Omega\times[0,T]\times\Lambda)$ by the same arguments as for $u^{g}$). Furthermore, we have

[TABLE]

since by the Lipschitz condition (3a) on $\sigma$ and the bound on the Fréchet derivative (3c) of $\sigma$ we have the following bound:

[TABLE]

This completes the proof that $z_{\delta}$ converges to [math] in $L^{2}([0,T]\times\Omega;H^{1}(\Lambda))$ and in

$L^{2}(\Omega;C([0,T];L^{2}(\Lambda)))$. From the definition of $h\mapsto y^{h}$, it follows immediately that this is linear. Thus, for the Gâteaux differentiability it remains to show that $h\mapsto y^{h}$ is continuous. But this follows with the same arguments as in Proposition 2.1. ∎

As a corollary we get the following representation for the Gâteaux derivative of the cost functional.

Corollary 3.1**.**

*For every $h\in L^{6}([0,T]\times\Lambda)$, the cost functional

$J:L^{6}\left([0,T]\times\Lambda\right)\to\mathbb{R}$ is Gâteaux differentiable in the direction $h$ with Gâteaux derivative*

[TABLE]

where $y^{h}$ denotes the variational solution of the SPDE (37).

Proof.

Recall that the cost functional is given by

[TABLE]

where

[TABLE]

and

[TABLE]

Hence

[TABLE]

Let $g\in L^{6}\left([0,T]\times\Lambda\right)$ be fixed. For $h\in L^{6}([0,T]\times\Lambda)$, we get for the Gâteaux derivative of $I_{2}$

[TABLE]

On the other hand we get for the Gâteaux derivative of $I_{1}$

[TABLE]

Hence, by the chain rule, we get

[TABLE]

which, together with equation (64) and Proposition 3.1, completes the proof. ∎

Now we can state a necessary condition for $J$ to attain a minimum.

Theorem 3.1**.**

Let $J$ attain a (local) minimum at $g^{\ast}\in G_{\text{ad}}$. Then, for every $h\in G_{\text{ad}}$ we have

[TABLE]

Proof.

Let $h\in G_{\text{ad}}$, and set $\delta_{t}:=g^{\ast}+t(h-g^{\ast})\in G_{\text{ad}}$. Since $g^{\ast}$ is a local minimizer, there exists a $t_{0}>0$ such that for all $t\in(0,t_{0})$ we have

[TABLE]

This implies

[TABLE]

Letting $t$ tend to zero yields the claim. ∎

4 The Gradient of the Cost Functional

In this section, we are going to derive a representation for the gradient of the cost functional via adjoint calculus. Recall the state equation

[TABLE]

In Section 3, we proved the following representation

[TABLE]

where $y^{h}$ is the variational solution of

[TABLE]

The adjoint equation that is used in the existing literature is

[TABLE]

for some processes $p\in L^{2}(\Omega\times[0,T];V)$ and $P\in L^{2}(\Omega\times[0,T];L_{2}(U,H))$. The derivation of the Stochastic Minimum Principle with this adjoint equation works in our setting as well. However, the numerical approximation of the solution of this adjoint equation is extremely costly. Therefore, we restrict our analysis to the case of additive noise in the state equation.

[TABLE]

for some $\sigma\in\mathbb{R}$. In this case, the linearized equation becomes

[TABLE]

which is a random partial differential equation (the coefficient $f^{\prime}(u^{g}_{t})$ is random). Now, we introduce the following random backward PDE for the adjoint state.

[TABLE]

One crucial point for our algorithm is the fact that the adjoint equation is a random backward PDE. The following property of the adjoint state is the main ingredient in the derivation of the gradient of the cost functional.

Lemma 4.1**.**

Let $p$ be the solution of the adjoint equation (76) and let $y^{h}$ be the solution of equation (75) associated with $u^{g}$. Then we have almost surely for every $h\in L^{6}([0,T]\times\Lambda)$

[TABLE]

Proof.

By the deterministic integration by parts formula, it holds pathwise

[TABLE]

Plugging in equations (75) and (76), respectively, this yields

[TABLE]

Integrating over $\Lambda$, and integrating the Laplace operator by parts, we get

[TABLE]

which is the claimed result. ∎

As a corollary, we get the following representation for the gradient of the cost functional.

Theorem 4.1**.**

The gradient of the cost functional is given by

[TABLE]

where $p$ is the solution of the adjoint equation

[TABLE]

Proof.

By Corollary 3.1, we have

[TABLE]

where $y^{h}$ denotes the variational solution of the random PDE (37). Now, by Lemma 4.1, this yields

[TABLE]

which completes the proof. ∎

Furthermore, by plugging this representation into the necessary condition derived in Theorem 3.1, we get the Stochastic Minimum Principle.

Theorem 4.2**.**

Let $J$ attain a (local) minimum at $g^{\ast}\in G_{\text{ad}}$. Then, for every $h\in G_{\text{ad}}$ we have

[TABLE]

5 Nonlinear Conjugate Gradient Descent

Now that we have identified a representation for the gradient, we can apply a probabilistic nonlinear conjugate gradient descent method in order to approximate the optimal control. We are going to briefly sketch our algorithm here. For a survey of nonlinear conjugate gradient descent methods see [14].

Let the initial control $g_{0}\in L^{6}\left([0,T]\times\Lambda\right)$ be given, and fix an initial step size $s_{0}>0$ as well as a stopping criterion $\eta>0$. Then, the next control can be found as follows.

Solve the state equation

[TABLE]

for one realization of the noise. 2. 2.

Solve the adjoint equation

[TABLE]

with the data given by the sample of the solution of the state equation that was calculated in Step 1. 3. 3.

Repeat Step 1 and Step 2 to approximate

[TABLE]

via a Monte Carlo method. 4. 4.

The direction of descent is given by $d_{n}=-\nabla J(g_{n})+\beta_{n}d_{n-1}$, where $\beta_{n}=\frac{\|\nabla J(g_{n})\|}{\|\nabla J(g_{n-1})\|}$. (In the first step, $\beta_{1}=0$.) 5. 5.

Compute the new control via $g_{n+1}=g_{n}+s_{n}d_{n}$. 6. 6.

Accept or deny the new control: Again using a Monte Carlo method, we compare the costs under the new control with the costs under the old control. If the new control decreases the costs, we accept the new control and go back to Step 1. Otherwise, we decrease the step size $s_{n}=s_{n}/2$ and then go back to Step 5. (In our simulations, it has proven useful to accept the new control even if the costs are non-decreasing, once the step size gets too small, e.g. $s_{n}<10^{-4}$.) 7. 7.

Stop if $\|\nabla J(g_{n})\|<\eta$, otherwise reset the step size $s_{n}=s_{0}$ and go to step 1.

6 Application to Optimal Control of the Stochastic Schlögl Model

In this section we want to present the application of the algorithm that was introduced in Section 5 to the stochastic Schlögl model. We are going to investigate two examples. The first one is to control the speed and the direction of travel of the wave developing in the Schlögl model with additive noise; the second one is an example, where the optimal control of the deterministic system differs from the optimal control of the stochastic system. Corresponding results for the deterministic model can be found in the work by Buchholz et al. (see [2]).

6.1 Steering of a Wave Front

Let us first recall the Schlögl model. We consider the state equation

[TABLE]

with homogeneous Neumann boundary conditions, where $b\equiv 1$, and the nonlinearity is of the form $f(u)=ku(u-1)(a-u)$ for some $k>0$, $a\in(0,1)$, and $\sigma\in\mathbb{R}$, i.e. the state equation takes the form

[TABLE]

In our example, we choose the time-horizon $[0,15]$, the space $\Lambda=[0,20]$, $k=1$, and $a=39/40$. These choices lead to two stable steady states, $u\equiv 0$ and $u\equiv 1$. As initial condition we choose

[TABLE]

In the deterministic case we get a traveling wave of the form $u^{0}(x+ct)$, where $c=\sqrt{2}(\frac{1}{2}-a)$ (cf. [6]). Figure 2 shows the solution in the deterministic case, and Figure 2 shows one realization of the solution in the stochastic case with $\sigma=0.5$.

We can see that the traveling wave slowly travels to the right. Our objective is now to first speed up the wave and then change the direction of travel. To this end, we consider the cost functional given by

[TABLE]

where $c_{\overline{\Lambda}}=1$, and the reference profile $u_{\overline{\Lambda}}$ is given by

[TABLE]

for $(t,x)\in[0,T]\times\Lambda$.

With the algorithm from Section 5 we can approximate the optimal control. Let us apply the algorithm to the stochastic case with $\sigma=0.5$, the stopping criterion $\eta=0.05$ and $100$ Monte Carlo simulations for the approximation of the gradient. One realization of the solution with applied optimal control is displayed in Figure 4. Figure 4 shows the corresponding optimal control.

6.2 Comparison with the Control of the Deterministic System

Simulations show that the optimal control for the deterministic system in the preceding example does not differ qualitatively from the optimal control for the stochastic system. This is because the fixed points [math] and $1$ are stable. The situation changes, however, if one of the fixed points becomes unstable from one side, as the following example shows. Consider the state equation

[TABLE]

where $\Lambda=[0,20]$, $T=30$ and $\sigma\in\mathbb{R}$. These choices lead to only one stable steady state, $u=1$ and one unstable steady state $u=0$. Now, as initial condition, we choose $u^{g}_{0}=0$, and consider the cost functional

[TABLE]

i.e., we want the final state to be unchanged, in the unstable steady state [math]. In the deterministic case, the optimal control is clearly $g^{\ast}=0$, since we start in the steady state $x=0$ and without any forcing, we stay in this state and accomplish the minimal possible costs $J(g^{\ast})=0$. In the stochastic case, however, the noise term pushes the state out of the unstable steady state. Whenever the noise pushes the state above [math], the dynamics of the state equation force the state towards the stable steady state $x=1$. As an illustration of this effect, Figure 6 displays the potential $F(x)$ of the nonlinearity $f$. Figure 6 shows one realization in the stochastic case without a control function.

When we introduce a control, the control tries to counteract this effect by keeping the state below [math] for times $t<T$. This effect can be seen in the simulations, as well. As the stopping criterion we used $\eta=0.002$. Figures 8 to 8 display the optimal controls in the stochastic case with $\sigma=0.5$ and one realization of the corresponding state.

6.3 Mathematical Analysis in a Simplified Setting

Since we are not able to prove the previous result in that setting rigorously, we consider a simpler similar example in which the optimal control in the deterministic case and the optimal control in the stochastic case differ.

Let us consider the stochastic ordinary differential equation

[TABLE]

where $(B_{t})_{t\geq 0}$ is a Brownian motion on $\mathbb{R}$, the potential $V:\mathbb{R}\to\mathbb{R}$ is given by

[TABLE]

and hence $-V^{\prime}$ is given by

[TABLE]

Notice that this potential qualitatively resembles the potential used in the previous example in the interval $[0,1]$. That is why we observe a similar effect in this example. We consider the cost functional

[TABLE]

As in the previous example, the initial condition and the desired final state are both the unstable steady state $u=0$. Hence, in the deterministic case ($\sigma=0$), the optimal control is given by $g^{\ast}\equiv 0$, since the constant function $u\equiv 0$ solves the deterministic equation without control and the associated costs are zero.

Now, we are going to show that the optimal control in the stochastic case ($\sigma>0$), however, is not equal to zero. First, notice that the adjoint equation associated with our control problem is given by

[TABLE]

where $-V^{\prime\prime}$ is given by

[TABLE]

Hence, the solution of the adjoint equation is given explicitly by

[TABLE]

and the gradient of the cost functional is given by

[TABLE]

Now, we are going to show that the gradient for $g\equiv 0$ is not equal to zero and hence, $g\equiv 0$ is not an optimal control. To this end, consider

[TABLE]

This yields

[TABLE]

where the last part is strictly positive since $u_{T}$ has a strictly positive density with respect to the Lebesgue measure. Therefore, the gradient is not equal to zero and thus, $g\equiv 0$ is not an optimal control.

Remark 6.1**.**

Notice that we did not use that $g\equiv 0$ in this proof. This shows, that the optimal control in the stochastic case is unbounded.

Figures 10 and 10 illustrate our results in case of the stochastic ordinary differential equation (96) as the constraint and the cost functional (99).

Acknowledgement

This work has been funded by Deutsche Forschungsgemeinschaft (DFG) through grant CRC 910 “Control of self-organizing nonlinear systems: Theoretical methods and concepts of application,” Project (A10) “Control of stochastic mean-field equations with applications to brain networks.” The authors would like to thank the referee for their valuable feedback.