Stabilization of Partial Differential Equations by Sequential Action Control

TL;DR

This paper introduces a sequential action control framework for stabilizing partial differential equations, utilizing variational principles and adjoint information, with applications demonstrated on an unstable heat equation.

Contribution

The paper develops a novel SAC framework for PDE stabilization, explicitly deriving control actions and analyzing closed-loop systems in Hilbert spaces.

Findings

Control actions can be explicitly obtained from variational principles.

The framework successfully stabilizes an unstable heat equation.

Numerical results verify the effectiveness of the approach.

Abstract

We present a framework of sequential action control (SAC) for stabilization of systems of partial differential equations which can be posed as abstract semilinear control problems in Hilbert spaces. We follow a late-lumping approach and show that the control action can be explicitly obtained from variational principles using adjoint information. Moreover, we analyze the closed-loop system obtained from the SAC feedback for the linear problem with quadratic stage costs. We apply this theory to a prototypical example of an unstable heat equation and provide numerical results as the verification and demonstration of the framework.

Tables

Table 1. Table 1 . Problem parameters chosen for the numerical study concerning the reaction-diffusion problem ( 54 ) and the chosen stage costs ( 55 )

Parameter	Meaning	Value
$L$	length of the spatial 1-D area	$1$
$\mu$	instability constant	$1.35{\pi }^2$
$y_0\left(x\right)$	initial value for the temperature profile	$\frac{1}{5}\sin \left(\pi x\right)$
$\overline{q}$	weight constant for matrix $Q$	$10$
$\beta$	constant	$1.6$
$t_s$	sampling time step	$0.1$

Equations

$$\dot{y}(t)=Ay(t)+f(t,y(t))u(t)\in H,t>0$$

$$y(0)=y_0$$

$$\min \text{leavevmode}{J}_1(u)=$$

$$\dot{y}(t)=Ay(t)+f(t,y(t))u(t),\text{leavevmode}t\in (0,T),y(0)=y_0,$$

$$u(t)\in U,\text{leavevmode}t\in (0,T),$$

$$u_{\lambda ,\tau ,v}=\{\begin{array}{ll}{u}_1(t),& t\notin [\tau -\frac{\lambda }{2},\tau +\frac{\lambda }{2}]\\ v,& t\in [\tau -\frac{\lambda }{2},\tau +\frac{\lambda }{2}].\end{array}$$

$$\frac{d{J}_1}{d{\lambda }^{+}}(\tau ,v)=\underset{\lambda \downarrow 0}{\lim }\frac{J_1(u_{\lambda ,\tau ,v})-J_1(u_1)}{\lambda }$$

$$u_{\text{opt}}(\tau )$$

$$\dot{y}(t)=Ay(t)+f(t,y(t))u(t),\text{leavevmode}u(t)\in U,t\in (0,T),y(0)=y_0$$

$$y(t)=e^{tA}{y}_0+{\int }_0^t{e}^{(t-s)A}f(s,y(s))u(s)ds,t\in [0,T]$$

$$\dot{p}(t)$$

$$p(T)$$

$$p(t)=e^{(T-t)A^{\ast }}p(T)+{\int }_t^T{e}^{(s-t)A^{\ast }}(((f(s,y(s))u(s){)}_y{)}^{\ast }p(s)+(l_1{)}_y(y(s)))ds,t\in [0,T]$$

$$P(t):w\mapsto ⟨p(t),f(t,y(t))w{⟩}_{H^{\ast },H},t\in [0,T],$$

$$\Pi (t)=(P^{\star }(t)P(t)+R^{\star }{)}^{-1},t\in [0,T],$$

$$u_{\text{opt}}(\tau )=\Pi (\tau )[P^{\star }(\tau )P(\tau )u_1(\tau )+P^{\star }(\tau ){\alpha }_d]$$

$$\frac{d{J}_1}{d{\lambda }^{+}}(\tau ,u)=⟨p(\tau ),f(\tau ,y(\tau ))(u(\tau )-u_1(\tau )){⟩}_{H^{\ast },H}$$

$$\frac{\mathfrak{d}{l}_2}{\mathfrak{d}{u}_{\text{opt}}}(u_{\text{opt}})\eta =\underset{\epsilon \downarrow 0}{\lim }\frac{1}{\epsilon }[l_2(u_{\text{opt}}+\epsilon \eta )-l_2(u_{\text{opt}})]=0.$$

$$l_2(u_{\text{opt}}(\tau )+\epsilon \eta (\tau ))=\text{leavevmode}$$

$$+\frac{1}{2}⟨u_{\text{opt}}(\tau )+\epsilon \eta (\tau ),R(u_{\text{opt}}(\tau )+\epsilon \eta (\tau )){⟩}_U$$

$$l_2(u_{\text{opt}}(\tau )+\epsilon \eta (\tau ))-l_2(u_{\text{opt}}(\tau ))=\text{leavevmode}$$

$$+\frac{1}{2}⟨p(\tau ),\epsilon f(y(\tau ),\tau )\eta (\tau ){⟩}_{H^{\ast },H}^2-{\alpha }_d⟨p(\tau ),\epsilon f(y(\tau ),\tau )\eta (\tau ){⟩}_{H^{\ast },H}$$

$$+\frac{\epsilon }{2}⟨u_{\text{opt}}(\tau ),R\eta (\tau ){⟩}_U+\frac{\epsilon }{2}⟨\eta (\tau ),R(u_{\text{opt}}(\tau )+\epsilon \eta (\tau )){⟩}_U.$$

$$\left[⟨p(\tau ),f(\tau ,y(\tau ))(u_{\text{opt}}(\tau )-u_1(\tau )){⟩}_{H^{\ast },H}-{\alpha }_d\right]⟨p(\tau ),f(\tau ,y(\tau ))\xi {⟩}_{H^{\ast },H}+⟨R^{\star }{u}_{\text{opt}}(\tau ),\xi {⟩}_U=0.$$

$$\left[⟨p(\tau ),f(\tau ,y(\tau ))(u_{\text{opt}}(\tau )-u_1(\tau )){⟩}_{H^{\ast },H}-{\alpha }_d\right]⟨p(\tau ),f(\tau ,y(\tau ))\xi {⟩}_{H^{\ast },H}+⟨R^{\star }{u}_{\text{opt}}(\tau ),\xi {⟩}_U$$

$$=⟨P(\tau )(u_{\text{opt}}(\tau )-u_1(\tau ))-{\alpha }_d,P(\tau )\xi {⟩}_{\mathbb{R}}+⟨R^{\star }{u}_{\text{opt}}(\tau ),\xi {⟩}_U$$

$$=⟨P^{\star }(\tau )(P(\tau )(u_{\text{opt}}(\tau )-u_1(\tau ))-{\alpha }_d),\xi {⟩}_U+⟨R^{\star }{u}_{\text{opt}}(\tau ),\xi {⟩}_U$$

$$=⟨(P^{\star }(\tau )P(\tau )+R^{\star })u_{\text{opt}}(\tau )-P^{\star }(\tau )P(\tau )u_1(\tau )-P^{\star }(\tau ){\alpha }_d,\xi {⟩}_U.$$

$$(P^{\star }(\tau )P(\tau )+R^{\star })u_{\text{opt}}(\tau )-P^{\star }(\tau )P(\tau )u_1(\tau )-P^{\star }(\tau ){\alpha }_d=0.$$

$$⟨(P^{\star }(\tau )P(\tau )+R)u,u{⟩}_U=(P(\tau )u{)}^2+⟨Ru,u{⟩}_U\ge {\gamma }_R\Vert u{\Vert }_U^2.$$

$$\frac{d{J}_1}{d{\lambda }^{+}}(\tau ,u_{\text{opt}}(\tau ))<0$$

$$\displaystyle\frac{dJ_{1}}{d\lambda^{+}}(\tau,u_{\text{opt}}(\tau))=\langle p(\tau),f(\tau,y(\tau))\big{(}\Pi(\tau)[P^{\star}(\tau)P(\tau)u_{1}(\tau)+P^{\star}(\tau)\alpha_{d}]-u_{1}(\tau)\big{)}\rangle_{H^{*},\;H}$$

$$=⟨p(\tau ),f(\tau ,y(\tau ))\Pi (\tau )P^{\star }(\tau ){\alpha }_d{⟩}_{H^{\ast },H}-⟨p(\tau ),f(\tau ,y(\tau ))u_1(\tau ){⟩}_{H^{\ast },H}$$

$$+⟨p(\tau ),f(\tau ,y(\tau ))\Pi (\tau )P^{\star }(\tau )P(\tau )u_1(\tau ){⟩}_{H^{\ast },H}$$

$$\displaystyle\quad\quad\quad\quad\quad\quad\leq P(\tau)\Pi(\tau)P^{\star}(\tau)\alpha_{d}+\|p(\tau)\|_{H^{*}}\bar{f}\|u_{1}(\tau)\|_{U}\big{(}\|\Pi(\tau)\|_{op}\bar{P}+1\big{)}.$$

$$\displaystyle|P(\tau)\Pi(\tau)P^{\star}(\tau)\alpha_{d}|<\|p(\tau)\|_{H^{*}}\bar{f}\|u_{1}(\tau)\|_{U}\big{(}1+\|\Pi(\tau)\|_{op}\bar{P}\big{)},$$

Full text

1 Department of Mathematics, Humboldt-Universität zu Berlin, Unter den Linden 6, 10099 Berlin, Germany (yan.brodskyi@hu-berlin.de, falk.hante@hu-berlin.de); 2Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany (arno.seidel@fau.de)

1. Introduction

With the rising performance of embedded and networked systems the stabilization of processes governed by partial differential equations (PDEs) in real time and under uncertainties becomes an important topic in control theory. Here we consider processes predicted by semilinear PDEs of evolution type with distributed control given in an abstract sense by the operator differential equation

[TABLE]

with a state $y$ and control $u$ in possibly infinite-dimensional spaces $H$ and $U$, respectively, and satisfying the initial condition

[TABLE]

for some initial state $y_{0}\in H$. The goal is to guarantee asymptotic stabilization of $y$ at the origin, or more generally, to consider nonlinear path following for a given desired trajectory $y_{d}(t)$, $t>0$. Similar problems also arise in optimal control of PDEs, if one wishes to exploit the Turnpike property, saying that in large optimization time horizons, the optimal solution remains exponentially close to an optimal stationary solution for most of the time. This property has been proven for various optimal control systems, see Grüne et al. (2019); Gugat and Hante (2019); Trélat and Zhang (2018) and the references therein.

Our investigations here concern a moving horizon strategy for the control design which allows to include measurements to account for uncertainties in an online fashion, e.g., for the typically not exactly known initial state $y_{0}$ for the next horizon. A challenging application, for instance, is the stabilization of gas networks, where the realized demand has some variations over a predicted one during intra-day operation, see e.g., Hante et al. (2017).

In this context, the most prominent control design is (nonlinear) model predictive control (MPC), where future control action is obtained from the solution of a dynamic optimization problem. Concerning problems involving PDEs, stability analysis for the closed loop with a focus on late lumping is carried out, for example, in Alla and Volkwein (2015); Altmüller (2014); Altmüller and Grüne (2012); Dubljevic et al. (2006); Fleig and Grüne (2018); Ito and Kunisch (2002); Grüne et al. (2019). In this work, we consider a moving horizon strategy differing from MPC in avoiding to numerically compute the solution to a time-depend optimal control problem in each step. Instead, a piecewise constant control action is computed by selecting a control value and an application time that ensures a certain decrease of the current stage costs. This principle has been introduced for nonlinear problems with ordinary differential equations as sequential action control (SAC) in Ansari and Murphey (2016). It relies on representations of the so-called mode insertion gradient in Axelsson et al. (2008) using adjoint information originating in optimization of switched dynamical systems and the needle variation in Pontryagin et al. (1962) as used in the derivation of Pontryagin’s principle.

Our contribution here is to extend the idea of sequential action control to the above PDE setting in a Hilbert space framework. We show that the control action can be explicitly obtained from variational principles using an adjoint PDE. Moreover, for the linear case this allows us to show that the closed-loop performance for quadratic stage costs can in first order be characterized by a system under a particular linear feedback. Using this, we derive mesh independent stabilizing properties of this method prototypically for a benchmark problem from Altmüller and Grüne (2012). A finite-dimensional controller can then be obtained using Galerkin approximations. Unlike applying the theory of Ansari and Murphey (2016) to a finite-dimensional (lumped) approximation of the PDE this allows independent discretizations for the forward and the adjoint problem. For this linear problem we also provide a numerical study for both full domain and subdomain control with the most important parameters in this framework. Further, we numerically investigate the robustness of the closed-loop stabilization by SAC with respect to random disturbances and compare SAC with the standard linear-quadratic regulator (LQR) method. SAC turns out to be faster in all cases and more robust for the case of subdomain control with small disturbances.

The remaining article is organized as follows. In Section 2, we introduce the Hilbert space framework of SAC. In Section 3, we consider the important special case of linear quadratic problems, for which we characterize the closed-loop system for the stability analysis of SAC actions. Moreover, we analyze the performance of SAC for an unstable heat equation. In Section  4, we show the corresponding numerical results including the comparison with LQR. In Section 5, we provide concluding remarks.

2. A Hilbert space framework for SAC

Let $H$ and $U$ be real (separable) Hilbert spaces with inner products $\langle\cdot,\cdot\rangle_{H}$ and $\langle\cdot,\cdot\rangle_{U}$, and its topological dual spaces $H^{*}$ and $U^{*}$, respectively. Further, let $\mathfrak{L}(U,H)$ denote the space of bounded linear operators from $U$ to $H$. In this setting we consider a moving horizon strategy with the stage problem

[TABLE]

where $A$ is a (possibly unbounded) linear operator on $H$, $l_{1}$ and $m$ are (possibly nonlinear) functionals on $H$, $f:(0,T)\times H\to\mathfrak{L}(U,H)$, where $\mathfrak{L}(U,H)$ is space of bounded linear operators, $y_{0}$ is a fixed look ahead initial state in $H$, $T$ is a fixed time horizon and the minimization is with respect to all $u\in U_{t}=L^{p}(0,T;U)$ for some $p\geq 1$.

In order to fix notation in what follows, we denote, whenever $X$ and $Y$ are Hilbert spaces, for $x\in X$ the dual pairing by $\langle x^{*},x\rangle_{X^{*},\;X}=x^{*}(x)$, and for any bounded linear operator $B$ from $X$ to $Y$ by $\|B\|_{op}=\inf\{c:\|Bx\|_{Y}\leq c\|x\|_{X}\leavevmode\nobreak\ \mbox{for all}\leavevmode\nobreak\ x\in X\}$ the operator norm, by $B^{*}\colon Y^{*}\rightarrow X^{*}$ defined as $\langle B^{*}(y^{*}),x\rangle_{X^{*},\;X}=\langle y^{*},Bx\rangle_{Y^{*},\;Y}$ the topological adjoint operator, and by $B^{\star}\colon Y\rightarrow X$ defined as $\langle B^{\star}y,x\rangle_{X}=\langle y,Bx\rangle_{Y}$ the Hilbert space adjoint operator. Further notations and technical assumptions on the operators and functionals in (1) will be introduced in the sequel.

2.1. The SAC principle

Given a reference control $u_{1}\in U_{t}$ ($u_{1}\equiv 0$ is a feasible choice), let $u_{\lambda,\tau,v}$ denote the needle variation defined by

[TABLE]

Further, let $\frac{dJ_{1}}{d\lambda^{+}}(\tau,v)$ denote the sensitivity

[TABLE]

for the current stage (1). The principle of SAC relies on choosing the control values $u_{\text{opt}}(\tau)$ as

[TABLE]

for some application time $\tau\in(0,T)$, where $\alpha_{d}<0$, is to be chosen appropriately in order to obtain a sufficiently large reduction for the current prediction according to (1) when $u_{\text{opt}}(\tau)$ is applied for the current stage on the interval $[\tau-\frac{\bar{\lambda}}{2},\tau+\frac{\bar{\lambda}}{2}]$ with a suitably chosen duration $\bar{\lambda}>0$ until a new control has been computed for a shifted prediction horizon. Here $\bar{\lambda}$ corresponds to the length of the application time of the control $u_{\text{opt}}(\tau)$. The operator $R$ in (3) can be suitably chosen as a regularization parameter or to model control costs in this process.

Figure 1 shows the schematic overview of the SAC principle. In general, it consists from the following steps:

1. (1)

predict the nominal dynamics of the state and the adjoint for the reference control $u_{1}$; 2. (2)

compute the control values $u_{\text{opt}}(\tau)$ according to the SAC principle (3); 3. (3)

select an application time $\tau$ and the action’s duration $\bar{\lambda}$; 4. (4)

apply $u_{\bar{\lambda},\tau,u_{\text{opt}}(\tau)}$ and repeat the process at the next sample time, $t_{0}=t_{0}+t_{s}$.

For details concerning all of these steps we make the following hypotheses.

Assumption 1**.**

The operator $A$ is densely defined on $D(A)\subset H$ and generates a strongly continuous semigroup $S(t)=e^{tA}$ on $H$. The cost functionals $l_{1}\colon H\to\mathbb{R}$ and $m\colon H\to\mathbb{R}$ are continuously differentiable in the sense of Fréchet and uniformly bounded. For every $t\in(0,T),\leavevmode\nobreak\ f(t,\cdot)\colon H\to\mathfrak{L}(U,H)$ is continuously Fréchet differentiable. Furthermore, $f$ is linearly bounded or $T$ is sufficiently small. The operator $R:U\to U$ is bounded, self-adjoint and satisfies, for some $\gamma_{R}>0$, the coercivity estimate $\langle Ru,u\rangle\geq\gamma_{R}\|u\|_{U}^{2}$ for all $u\in U$.

Under the above assumptions, the abstract initial value problem in (1)

[TABLE]

has a unique solution $y\in C([0,T];H)$ in the mild sense, i.e.,

[TABLE]

for any $u\in U_{t}$ and any $y_{0}\in H$. Moreover, the following adjoint problem

[TABLE]

also has a unique solution $p\in C([0,T];H^{*})$ in a mild sense, i.e.,

[TABLE]

for any $u\in U_{t}$. Here $f(t,y(t))u(t))_{y}$ is a gradient of $f(t,y(t))u(t))$ with respect to $y$. $A^{*}$ is the generator of $C_{0}$-semigroup on $H^{*}$, see (Pazy, 1983, Chapter 1, Theorem 10.6). The existence and uniqueness of the mild solution of (6) is given by (Pazy, 1983, Chapter 6, Theorem 1.2) due to the continuous Fréchet differentiability assumptions on $f$ and linearity of the adjoint problem in $p$. In addition, the problem (3) admits an explicit solution, as stated in the following theorem.

Theorem 2.1**.**

Let $y\in C([0,T];H)$ be a solution of the state equation in (1) with $u$ being a given reference control $u_{1}\in C(0,T;U)$. Further, let $p\in C([0,T];H^{*})$ be the corresponding solution of the adjoint problem (6). With $P\in C([0,T];U^{*})$ defined by

[TABLE]

for which the operator $P^{\star}(t)P(t)+R^{\star}$ has a bounded inverse. And with

[TABLE]

the optimal solution $u_{\text{opt}}(\tau)$ of the problem (3) is given by

[TABLE]

for all $\tau\in[0,T]$.

Proof.

Under the Assumption 1, we can use results in (Rüffler and Hante, 2016, Theorem 10) to obtain that the limit in (2) exists and that it is given by

[TABLE]

with $p$ satisfying (6). Moreover, as a sum of convex functions, $l_{2}(\tau,u(\tau))$ is convex as a function of $u$. With this, necessary optimality conditions become sufficient. Classical optimality conditions ((Hinze et al., 2009, Theorem 1.46)) now yield that

[TABLE]

With (11), we obtain that for all $\tau\in[0,T]$ and all $\eta\in C(0,T;U)$

[TABLE]

and, with a short calculation, that

[TABLE]

By linearity, the right hand side in (13) converges uniformly as $\varepsilon\to 0$. Hence, we can take the limit in (12) and obtain, for almost every $\tau\in[0,T]$, for all $\xi\in U$

[TABLE]

With $P$ defined as in (8), we have

[TABLE]

Using (15) in (14), and that it holds for all $\xi\in U$ yields

[TABLE]

For all $\tau\in[0,T]$ the operator $P^{\star}(\tau)P(\tau)$ is self-adjoint and under the hypotheses in Assumption 1 the operator $P^{\star}(\tau)P(\tau)+R^{\star}$ is self-adjoint and satisfies

[TABLE]

Hence, the operator $P^{\star}(\tau)P(\tau)+R^{\star}$ has a bounded inverse for all $\tau\in[0,T]$ and (10) follows by rearranging terms in (16). ∎

The computation of the SAC actions using the adjoint as in Theorem 2.1 is highly efficient compared to the computation of the SAC actions using the numerical approximations of the sensitivity.

2.2. Action duration

For $u_{\text{opt}}(\tau)$ given by the SAC principle (3), for arbitrary time $\tau\in(0,T)$ we now want to assure that we can find the duration $\bar{\lambda}$, such that the corresponding change in cost $J_{1}$ will be negative. Since this is not obvious in the infinite dimensional setting, we provide an explicit proof.

Proposition 2.2**.**

Let $y\in C([0,T];H)$ be a solution of the state equation in (1), $p\in C([0,T];H^{*})$ be the corresponding solution of the adjoint problem (6) with $u_{1}\in C(0,T;U)$ being a given reference control. Furthermore, let $u_{\text{opt}}(\tau)$ be a control obtained by the SAC principle (3). Then there exists a parameter $\alpha_{d}<0$, such that the mode insertion gradient (2) satisfies

[TABLE]

for all $\tau\in(0,T)$.

Proof.

Combining (10), (11) and using Assumption 1, with the operator norms $\bar{f}=\|f(\tau,y(\tau))\|_{op}$ and $\bar{P}=\|P^{\star}(\tau)P(\tau)\|_{op}$ we have

[TABLE]

The operator $\Pi(\tau)$ is positive, as the inverse of a positive operator. This gives us that the operator $P(\tau)\Pi(\tau)P^{\star}(\tau)$ is also positive. By choosing $\alpha_{d}<0$ such that

[TABLE]

we receive $\frac{dJ_{1}}{d\lambda^{+}}(\tau,u_{\text{opt}}(\tau))<0$. ∎

Corollary 2.3**.**

Let $y\in C([0,T];H)$ be a solution of the state equation in (1), $p\in C([0,T];H^{*})$ be the corresponding solution of the adjoint problem (6) with $u_{1}\in C(0,T;U)$ being the given reference control. Furthermore, let $u_{\text{opt}}(\tau)$ be a control obtained by the SAC principle (3). Then for all $\tau\in(0,T)$ there exists a parameter $\alpha_{d}<0$ and there exists a neighborhood $N(\tau)\subset(0,T)$ around $\tau$, such that the mode insertion gradient (2) satisfies

[TABLE]

for all $t\in N(\tau)$.

Proof.

Continuity of the state $y(t)$, the corresponding adjoint $p(t)$, and $f(t,y)$ with respect to $t$ provided by Assumption (1) gives us

[TABLE]

[TABLE]

[TABLE]

Denote

[TABLE]

Furthermore, by Assumption 1, for every fixed $t\in[0,T]$, we have the expansion

[TABLE]

with $\|h\|_{H}\rightarrow 0$, $K(t)$: $H\to\mathfrak{L}(U,H)$ being a bounded linear operator. Moreover, denote the operator norms $\bar{P}=\|P^{\star}(\tau)P(\tau)\|_{op}$ and $\bar{f}=\|f(t,y(t))\|_{op}$. Combining (10) and (11) we receive

[TABLE]

The second part of the right-hand side can be bounded using the norms of the adjoint and the reference control

[TABLE]

The remaining part of (26) can be rearranged as

[TABLE]

One of the parts can be directly bounded using (21)

[TABLE]

The remaining part of (28) can be rewritten using (25)

[TABLE]

Last two terms of the right-hand side of (30) can be bounded using (23)

[TABLE]

Finally, the first term of (30) can be transformed and bounded using (22)

[TABLE]

Combining all previously obtained bounds (27),(29),(31) and (32) with $\epsilon$ defined in (24), we can bound the mode insertion gradient (2)

[TABLE]

Here we can see, that all the terms on the right-hand side of (33) which contain $|\alpha_{d}|$ also contain $\epsilon$, which can be made arbitrary small by the appropriate choice of $\delta$ defined in (24). This gives us that for all $\tau\in(0,T)$, there exist a neighborhood $N(\tau)\in(0,T)$ and $\alpha_{d}<0$ which has to satisfy the following inequality

[TABLE]

such that

[TABLE]

for all $t\in N(\tau)$. And this concludes the proof. ∎

The result in Corollary 2.3 gives us, that for every $\tau\in(0,T)$, for an appropriately chosen coefficient $\alpha_{d}$, there exists some length $\bar{\lambda}$ of application time of the control $u_{\text{opt}}(\tau)$, found by the SAC principle (3), such that for all $t\in[\tau-\frac{\bar{\lambda}}{2},\tau+\frac{\bar{\lambda}}{2}]$, $\frac{dJ_{1}}{d\lambda^{+}}(t,u_{\text{opt}}(\tau))\leq 0$. And by using (2) we receive that for every fixed point $t\in[\tau-\frac{\bar{\lambda}}{2},\tau+\frac{\bar{\lambda}}{2}]$:

[TABLE]

This means, that the change in the cost can be locally bounded from above, and controlled by the appropriately chosen $\alpha_{d}$ and $\bar{\lambda}$. In practice the action duration $\bar{\lambda}$ can be found using a standard line search method. We note that, in addition, practical versions of SAC take into account a small computation time $\Delta_{c}t$ needed to solve (10) numerically as well as additional steps such as computation of efficient application times $\tau$. Moreover, saturation techniques for control constraints can be used in each iteration. These steps can be implemented as proposed in Ansari and Murphey (2016) and shall therefore not be discussed here further.

2.3. Linear systems case

In this subsection we specialize the main results of the previous subsection to the case $f(t,y)=B$, with $B$ being a bounded control operator on a control space $U$ with images in $H$, that is, we consider now

[TABLE]

with the mild solution

[TABLE]

for any $u\in U_{t}$ and any $y_{0}\in H$. Moreover, the adjoint problem (6) simplifies to

[TABLE]

with the corresponding mild solution

[TABLE]

Using Theorem 2.1, we have that the problem (3) has an explicit solution given by

[TABLE]

for all $\tau\in[0,T]$, with $P\in C([0,T];U^{*})$ defined by

[TABLE]

Remark*.*

It is convenient to identify the Hilbert spaces $H^{*}$ and $U^{*}$ with $H$ and $U$, respectively. We identify $p\in C(0,T;H^{*})$ with $\pi\in C(0,T;H)$ so that

[TABLE]

Applying this to the adjoint problem (40), we receive the following problem in $H$

[TABLE]

where $(\tilde{l}_{1})_{y}(y(t))$ and $\tilde{m}_{y}(y(T))$ are unique Riesz representations of $(l_{1})_{y}(y(t))$ and $m_{y}(y(T))$ in H, respectively, and $A^{\star}$ is the Hilbert adjoint of the unbounded operator $A$.

For simplicity of the notations, we continue writing $p$ for $\pi$ in Sections 3 and 4.

3. The SAC feedback for linear quadratic problems

In this section we consider the important special case of quadratic stage costs. In this setting, one can derive a linear feedback law that describes in first order the dynamics of the closed loop system when controls given by the SAC-principle are continuously applied as the stage problem is shifted from $[0,T]$ to $[t,t+T]$, with $t>0$.

3.1. The SAC feedback for quadratic stage costs

Our analysis concerns the case of quadratic stage costs of the form

[TABLE]

subject to (37) with $y_{d}=0$, $u_{1}=0$, $Q$ being a bounded linear operator on $H$ and $P_{T}$ a positive and self-adjoint operator on $H$. In the following, we identify the Hilbert spaces $H^{*}$ and $U^{*}$ with $H$ and $U$, respectively (as described in the Remark Remark).

Lemma 3.1**.**

Let $F(\tau)\colon H\to H$ be a self-adjoint, positive, bounded linear operator and the mild solution of the differential Lyapunov equation

[TABLE]

Then for quadratic costs of the form (44), it holds

[TABLE]

Proof.

For (44) $m_{y}(y(t+T))=P_{T}y(t+T)$ and $(l_{1})_{y}(y(\tau))=Q^{\star}Qy(\tau)$. By (39) and (45) it holds that $p(t+T)=P_{T}y(t+T)=F(T)y(t+T)=F(t+T-t)y(t+T)$. Furthermore, for any $y\in L^{2}(t,t+T;H)$, a mild solution of (45) is given by

[TABLE]

see (Bensoussan et al., 2007, Part IV, Section 2). Now, rewriting (40) using (44) and (38) with control $u_{1}=0$, we obtain

[TABLE]

[TABLE]

Equality of the right-hand-sides of (48) and (49) along with the initial conditions gives us (46). ∎

Using the Lemma above and taking $p(\tau)$ at point $t$, with $\overline{F}=F(0)$ we have

[TABLE]

[TABLE]

Furthermore, incorporating $u_{\text{opt}}$ given by (41) into (37) we receive

[TABLE]

Theorem 3.2**.**

For (44), the following statements hold.

1. (1)

The SAC action satisfies

[TABLE] 2. (2)

The nonlinear operator G from (50) is Fréchet differentiable. 3. (3)

A linearization at the equilibrium of a continuous application of controls computed by SAC is a system under linear feedback

[TABLE]

Proof.

1. (1)

From (8), we have that for all $v,w\in U$

[TABLE]

and

[TABLE]

Hence, the self-adjoint operator $P^{\star}(t)P(t)$ depends quadratically on $y$. Moreover, using that $F$ as a solution of (45) is independent of $y$, we have

[TABLE]

for some constants $C_{B^{\star}},C_{F}>0$. The inverse operator (10) can be written as

[TABLE]

With the above, we get from (41)

[TABLE]

for $\|y\|_{H}$ sufficiently small. Using (42) in (53) yields (51). 2. (2)

Now, we consider the nonlinear part of (50), $G(y)=B[(P^{\star}(t)P(t)+R^{\star})(y)]^{-1}B^{\star}\bar{F}y(t)\alpha_{d}$. The operator $(P^{\star}(t)P(t)+R^{\star})(y):u\in U\mapsto\alpha_{d}\langle B^{\star}\bar{F}y,u\rangle_{U}B^{\star}\bar{F}y+R^{\star}u\in H$ is Fréchet differentiable with respect to $u\in U$, furthermore, the operator has a bounded inverse (see Th. 2.1). Thus, using the Inverse Function Theorem (see Serovajsky (2013)), the inverse operator $[(P^{\star}(t)P(t)+R^{\star})(y)]^{-1}$ is Fréchet differentiable as an operator on $u$.

Furthermore, for any fixed $u\in U$, $(P^{\star}(t)P(t)+R^{\star})(y)u=\alpha_{d}\langle B^{\star}\bar{F}y,u\rangle_{U}B^{\star}\bar{F}y+R^{\star}u$ is Fréchet differentiable as a function of $y$. So due to the invertibility of the operator $(P^{\star}(t)P(t)+R^{\star})(y)$, using (Potthast, 1994, Theorem 2) we get that $[(P^{\star}(t)P(t)+R^{\star})(y)]^{-1}u$ is Fréchet differentiable as a function of $y$.

Combining these results and the chain rule, we obtain that the operator $G(y)=B[(P^{\star}(t)P(t)+R^{\star})(y)]^{-1}B^{\star}\bar{F}y(t)\alpha_{d}$ is Fréchet differentiable as a function of $y$. 3. (3)

Using the result from the first statement, by using (53) in (37), we directly get a linearization at the equilibrium of a continuous application of controls computed by SAC, which is a system under linear feedback

[TABLE]

∎

For given $A,B,Q,P_{T},$ and $R$, the closed-loop system (52) can be analyzed for asymptotic and exponential stability. For instance, it is well known that if under the above settings, the operator $A+DG(0)$, which is the infinitesimal generator of a $C_{0}$-semigroup $T(t)$, satisfies

[TABLE]

then $y_{d}=0$ is exponentially asymptotically stable. See (Kato, 1995, Corollary 2.2.). In the parabolic case with asymptotic stability of the linearization, we can use results from (Henry, 1981, Theorem 5.1.1) to get asymptotic stability of the equilibrium of the original system in the appropriate space. We do this prototypically for a selected example concerning an unstable parabolic problem in the next subsection. Moreover, for reaction-diffusion equations and for wave equations, linearization and stability analysis can be done using the appropriate generalizations of the Hartman-Grobman theorem (see e.g., Lu (1991) and Rodrigues and Ruas-Filho (1997), Hein and Prüss (2016), respectively).

3.2. Unstable heat equation

In this subsection, we consider the stabilization of the one-dimensional reaction-diffusion process

[TABLE]

at $y_{d}=0$ with the quadratic stage costs

[TABLE]

for real constants $\beta,\bar{q}>0$, $\chi_{(a,b)}(x)=1$, for $x\in(a,b)$ and [math] otherwise, $\mu>\left(\frac{\pi}{L}\right)^{2}$, $(a,b)\subset(0,L)$ with the SAC framework of Section 2. Here we take $\mu$ larger then the smallest eigenvalue of $(-\Delta)$. We note that the solution of (54) without control, i.e., $u(t)=u_{1}(t)=0$, is exponentially unstable. The stabilization of (54) with classical MPC schemes was investigated in Altmüller and Grüne (2012).

With the spaces $H=L^{2}(0,L)$, $U=L^{2}(0,L)$, the operators $A$ and $B$ defined by $Ay=\mu y+\Delta y$ for $y\in D(A)=H_{0}^{1}(0,L)\cap H^{2}(0,L)$ and $Bu=\sqrt{\beta}u$ for $u\in U$, the control problem (54) can be written as

[TABLE]

see, e.g., Bensoussan et al. (2007). With $P_{T}=0$, $Qy=\bar{q}y$, the cost function (55) has the quadratic form (44). Moreover, for the SAC principle, we choose $R$ in (3) as the identity in $H$. Then, the closed-loop system (52) becomes

[TABLE]

with $\overline{F}_{q}$ as a solution of the Riccati equation (45) given by

[TABLE]

from (47) and using that $A$ is self-adjoint.

In the following we analyze the closed-loop system (56) with control $u$ on the full space $(a,b)=(0,L)$. We provide computational studies in Section 4 for the control on both full and subdomain. First, we characterize solutions of the closed-loop system (56) using a product approach.

Lemma 3.3**.**

Let $\phi_{k}$, $k=1,...,\infty$, be eigenfunctions, $D_{k}$ being eigenvalues of the Dirichlet-Laplace operator $\Delta y$ on $D(A)\subset H$ and assume that $\mu\neq-D_{k}$ for all $k\in\mathbb{N}$. Furthermore, let

[TABLE]

with $\alpha_{k}(0)=\chi_{k}$ and constants $\chi_{k}:=\langle y_{0},\phi_{k}\rangle_{L^{2}(0,L)}$, $k=1,...,\infty$. Then, the solution $y$ of (56) is within the set of functions

[TABLE]

for which the sum converges.

Proof.

In the operator form the semigroup $e^{TA}$ acts on the function $\phi_{k}(\cdot)$ as $e^{T(\mu+D_{k})}\phi_{k}(\cdot)$, see, e.g., Engel and Nagel (2000). Hence, using (57) and the ansatz (58), we obtain from (56) the equation

[TABLE]

Here, $\phi_{k}$ represents the spectral decomposition of $\phi$ in $L^{2}(0,L)$. Now dividing (59) by $\phi_{k}(x)$ and substituting $\frac{\phi_{k}^{\prime\prime}(x)}{\phi_{k}(x)}$ by $D_{k}$ we receive a first order ODE

[TABLE]

By computing the integral explicitly and rearranging terms, we get

[TABLE]

Now define $\delta_{k}=(\mu+D_{k})$ and $\kappa_{k}=\frac{\alpha_{d}\beta\bar{q}^{2}}{2(\mu+D_{k})}$. With this, we obtain

[TABLE]

Using the initial value $\alpha_{k}(0)=\chi_{k}$ we can solve (61):

[TABLE]

[TABLE]

So the solution of the equation (60) is given by

[TABLE]

Using that $\phi_{k}$, $k=1,\ldots,\infty$, is a basis of $H$ concludes the proof. ∎

The following result concerns the asymptotic stability of the closed-loop system (56) for $t\to\infty$ in dependency of the most important parameters $T,\bar{q},\beta$ and $\alpha_{d}$ of the SAC-principle.

Theorem 3.4**.**

Let $D_{k}$ being the eigenvalues of the Dirichlet-Laplace operator $\Delta y$ on $D(A)\subset H$ and assume that $\mu\neq-D_{k}$ for all $k\in\mathbb{N}$. Then for any $T,\bar{q},\beta>0$, there exists $\bar{\alpha}_{d}<0$ such that the closed-loop system (56) is asymptotically stable in $y_{d}=0$ for any $\alpha_{d}\leq\bar{\alpha}_{d}$.

Proof.

We use the characterization of solutions to the closed-loop system (56) from (3.3). From $\|y(t,x)\|_{H}\leq\sum_{k=1}^{\infty}|\alpha_{k}(t)|\|\phi_{k}\|_{H}$, we want to show that $|\alpha_{k}(t)|\to 0$ uniformly in $k$ for $t\to\infty$. To this end, and noting that, for all $k=1,\ldots,\infty$, the coefficients $\alpha_{k}$ are continuously differentiable for all $t$, we take a look on the derivative of $\alpha_{k}$

[TABLE]

The term $\frac{\dot{\alpha}_{k}(t)}{\alpha_{k}(t)}$ must be less than zero, so that the function $|\alpha_{k}(\cdot)|$ decreases close to zero. To guarantee this we require

[TABLE]

where $C=-\min\{|\mu+D_{k}|:k\in\mathbb{N}\}$ is independent of $k$.

Consider we first the case when $(\mu+D_{k})<0$. Then reorganizing (62) and multiplying both sides by $2(\mu+D_{k})$ we obtain

[TABLE]

From $(\mu+D_{k})<0$, we have that $\left(e^{2T(\mu+D_{k})}-1\right)<0$ and obtain the condition

[TABLE]

This inequality holds for any $\alpha_{d}<0$.

Now we consider the second case when $\mu+D_{k}>0$. In order to have under this condition the inequality $\left(e^{2T(\mu+D_{k})}-1\right)>0$, we obtain from (62) an explicit inequality constraint for $\alpha_{d}$ for different $k$

[TABLE]

Hence, for each $k=1,\ldots,\infty$ we can find appropriate $\alpha_{d,k}$, under conditions (63), for which asymptotic stability holds.

Since there are only finitely many $(\mu+D_{k})>0$, we can always find an $\alpha_{d}$ which implies (62) for all $k=1,\ldots,\infty$ and for which $\|y(t)\|_{H}$ is decreasing. For this we can for example choose

[TABLE]

This completes the proof. ∎

Theorem 3.4 together with (Henry, 1981, Theorem 5.1.1), in which we have for our case $X=L^{2}(0,L)$ and $X^{\alpha}=X^{\frac{1}{2}}=H^{1}_{0}(0,L)$, gives us that the closed-loop system obtained after implementation of SAC is locally uniformly asymptotically stable at equilibrium in $X^{\alpha}=H^{1}_{0}(0,L)$.

Together with the results of Subsection 3.1, this yields that continuously applied SAC with any $T,\bar{q}>0$ and a sufficiently small $\alpha_{d}<0$ stabilizes (54) in $y_{d}=0$ if $\|y_{0}\|$ is sufficiently small. Our numerical experiments indeed reveal that this result does not extend to global asymptotic stability, i.e., in general, one cannot find a fixed $\alpha_{d}$ for which stability of the closed-loop is guaranteed for any $y_{0}\in Y$. Moreover, too small $\alpha_{d}$ result in very large control actions, so that a sufficiently short time stepping is needed for a numerical realization in order to avoid overshooting. This suggests choosing $\alpha_{d}$ depending on $y_{0}$, for example by setting $\alpha_{d}=\gamma J_{1}(u_{1})$ with some $\gamma<0$.

If we take $\mu=\left(\frac{\pi}{L}\right)^{2}$ or smaller, then any negative constant $\gamma$ or even small enough (by absolute value) negative $\alpha_{d}$ would lead to stabilization of the state. This indicates that SAC actions also qualify for rapid stabilization. The corresponding analysis on stabilization rates may be considered in future work.

In Subsection 4.2 we provide a parameter study for the choice of the constants $\gamma$ and $T$.

4. Discretization and numerical results

In this section we will provide some theoretical results concerning Galerkin approximations of SAC actions for linear PDEs of parabolic type and nonlinear costs along with the numerical results for the unstable heat equation under certain conditions. In particular, we investigate numerically the stabilization properties on SAC for control on a subdomain and the case of partial observations. Moreover, we see how disturbance in the instability constant will affect the results, and make a comparison of SAC with standard LQR method.

4.1. Galerkin approximations

In order to obtain a finite-dimensional controller, we consider here Galerkin approximations. On the level of the discretizations, we can then compare the proposed late-lumping control actions with those of Ansari and Murphey (2016) applied to a finite-dimensional approximation of the PDE. For this subsection, we identify the Hilbert spaces $H^{*}$ and $U^{*}$ with $H$ and $U$, respectively (as described in the Remark Remark). Furthermore, we will work under the assumptions that $A$ is induced by a bilinear form in the following settings:

1. (1)

$V\hookrightarrow H\simeq H^{*}\hookrightarrow V^{*}$ is a Gelfand triple, $V$ separable Hilbert space. 2. (2)

$a(\cdot,\cdot):V\times V\rightarrow\mathbb{R}$ is a bilinear form and there are $\alpha,\beta>0$ and $\gamma\geq 0$ with

[TABLE]

Under these settings and the Assumption 1, the corresponding operator $A\colon D(A)\to H$, densely defined by

[TABLE]

generates a $C_{0}$-semigroup (see (Dautray and Lions, 1992, Theorem 3, p.330)).

The mild solution $y\in C(0,T;H)$ of (37) and the mild solution $\pi\in C(0,T;H)$ of (43) then coincides with the weak solutions given by $\tilde{y}\in W(0,T;H,V)$ and $\tilde{p}\in W(0,T;H,V)$, respectively, for almost every $t\in[0,T]$

[TABLE]

[TABLE]

see Ball (1977), Bensoussan et al. (2007).

Using Remark Remark, we also have for almost every $t\in[0,T]$

[TABLE]

with $p\in C(0,T;H^{*})$ being the mild solution of (39).

Let $V_{h}^{1}\subset V$ be a finite-dimensional subspace of $V$ with a basis $(\phi_{i})_{i=1}^{N}$, $V^{2}_{h}\subset V$ be a finite-dimensional subspace of $V$ with a basis $(\kappa_{i})_{i=1}^{K}$ and $U_{h}\subset U$ be a finite-dimensional subspace of $U$ with basis $(\psi_{i})_{i=1}^{M}$. With the ansatz

[TABLE]

we get an approximation of (64) by

[TABLE]

with matrices

[TABLE]

see, e.g., Hinze et al. (2009).

Similar, with $\hat{l}_{1}\colon\mathbb{R}^{N}\to\mathbb{R}$ and $\hat{m}\colon\mathbb{R}^{N}\to\mathbb{R}$ defined as

[TABLE]

we get an approximation of (1) by

[TABLE]

and an approximation of (65) by

[TABLE]

with $\tilde{M}=(\langle\kappa_{j},\phi_{i}\rangle_{H})_{i=1\ldots,N,\leavevmode\nobreak\ j=1,\ldots,K}$, $\tilde{A}=(a(\phi_{j},\kappa_{i}))_{j=1,\ldots,N,\leavevmode\nobreak\ i=1,\ldots,K}$, and

[TABLE]

This yields the following late-lumping SAC control action.

Proposition 4.1**.**

For a discretized reference control $\hat{u}_{1}$ such that $u_{1}(\tau)=\sum_{i=1}^{M}\hat{u}_{1,i}(\tau)\psi_{i}$, $\hat{p}$ being a solution of (70), $\tilde{\Lambda}(\tau)=\tilde{B}\hat{p}(\tau)\hat{p}^{\top}(\tau)\tilde{B}^{\top}$ with $\tilde{B}=(\langle\kappa_{j},B\psi_{i}\rangle_{H})_{i=1,\ldots,M,\leavevmode\nobreak\ j=1,\ldots,K}$, and $\tilde{R}=(\langle\psi_{i},R\psi_{j}\rangle_{U})_{i,j=1,\ldots,M}$ an approximation of (41) is given by

[TABLE]

for all $\tau\in[0,T]$.

Proof.

Under the given spaces identifications, Remark Remark, and (66) we can rewrite (41) in the following form

[TABLE]

for all $\tau\in[0,T]$, with $P\in C([0,T];U^{*})$ defined by

[TABLE]

and with

[TABLE]

Plugging (67) into (73), we get

[TABLE]

[TABLE]

thus, we define the following approximation operator

[TABLE]

Applying (67) to the variational form of the implementation of operator $R$, we get $\tilde{R}$

[TABLE]

Plugging (77) and $\tilde{R}$ into (74) we get

[TABLE]

Now plugging (77) and (78) into (72) and using that $(\tilde{\Lambda}+\tilde{R}^{\top})$ is invertible as $\tilde{\Lambda}$ being symmetric positive semidefinite and $\tilde{R}$ being symmetric positive definite, the result follows. ∎

The alternative early-lumping approach is to apply the SAC principle from Ansari and Murphey (2016) directly to the discretized problem (69) subject to (68). With $\tilde{R}$ as in Proposition 4.1 a SAC action $\bar{u}_{\text{opt}}(\tau)$ is then chosen as

[TABLE]

For the discretized reference control $\hat{u}_{1}$ as in Proposition 4.1, we then get

[TABLE]

where $\Lambda(\tau)=\mathcal{B}^{\top}((\mathcal{M}^{-1})^{\top}\hat{\rho}(\tau))\hat{\rho}(\tau)^{\top}\mathcal{M}^{-1}\mathcal{B}$, $\hat{\rho}$ solves the backward ODE

[TABLE]

and $\hat{y}$ is the solution of (68). Only under certain assumptions it holds that the two approaches yield the same control action.

Proposition 4.2**.**

Choosing $V_{h}^{1}=V^{2}_{h}$ with the same basis, i.e., $K=N$ and $\phi_{i}=\kappa_{i}$ for $i=1,\ldots,N$, we have

[TABLE]

In particular, $\hat{u}_{\text{opt}}$ obtained from (71) and $\bar{u}_{\text{opt}}$ obtained from (79) coincide.

Proof.

Choosing $V_{h}^{1}=V^{2}_{h}$ with the same basis functions, we have $\mathcal{A}^{\top}=\tilde{A}$, $\mathcal{B}^{\top}=\tilde{B}$ and $\mathcal{M}^{\top}=\tilde{M}$. With that, the result follows from (70) and (80) as well as (71) and (79). ∎

Remark*.*

The motivation to use different Ansatz functions for the state and the adjoint state comes from the reason that the adjoint variable $p$ can admit more regularity than the state $y$. Thus, it is meaningful to keep different Ansatz functions. A detailed discussion can be found in (Hinze et al. (2009), Section 3.2).

We note that error analysis can be applied to (71) in order to estimate the error of the finite-dimensional SAC control action in terms of data for the stage problem (1). For appropriate techniques concerning parabolic problems, see, e.g., Meidner and Vexler (2008).

4.2. Numerical results

For our numerical study of SAC for the problem (54) we choose the parameters provided in Table 1. Our numerical implementation uses the Galerkin approximation presented in Subsection 4.1. Furthermore, we use a Gelfand triple with $H=L^{2}(\Omega),V=H^{1}_{0}(\Omega)$, and $V^{*}=H^{-1}(\Omega)$. We work under the hypothesis of Proposition 4.2 and choose piecewise linear functions for the finite-dimensional state and adjoint subspaces $V_{h}=V^{*}_{h}$. For the finite-dimensional control subspace $U_{h}$ we take piecewise constant functions on an equidistant grid with mesh size $h=0.01$. The resulting ODEs are solved numerically using the implicit Euler method in time at the sampling times $t_{s}$. Unlike in the original SAC algorithm in Ansari and Murphey (2016) and the generalization in Section 2, but in order to make the numerical results comparable to the theoretical results in the Subsection 3.2, we consider the calculation time $t_{calc}=0$ and a fixed control application time $\bar{\lambda}=t_{s}$. However, we note that our numerical experiments reveal that small $t_{calc}>0$ time stepping for $\bar{\lambda}$ using line search does not change the results qualitatively.

4.2.1. Full domain control.

We recall that the solution of (54) without control, i.e., $u(t)=u_{1}(t)=0$, is exponentially unstable. Our numerical stabilization results for the full domain control are reported in Figure 2. The Subfigures (A) and  (B) illustrate the performance of the finite-dimensional SAC controller for driving the state towards the unstable equilibrium $y_{d}=0$ with the choice $T=1$ and $\gamma=-0.5$. In Subfigure (C), we see that smaller $\gamma$ lead to faster stabilization. However, due to our fixed implicit time stepping, the rate is limited by overshooting which becomes visible in our example for $\gamma=-1$ in $t=0.5$. Subfigure (D) shows that a longer time horizon $T$ leads to a smaller error of the state in the $L_{2}$-norm. We can observe that the stabilization rate is actually exponential until the error drops below a small constant that depends on the chosen sampling time.

4.2.2. Subdomain control.

Coming back to more general settings, the results for control on subdomain $\Omega_{c}=[0,\infty)\times(0.5,0.9)$ are reported in Figure 3. Furthermore, in order to check robustness of the closed-loop system after incorporating controls obtained by SAC with respect to some random disturbance, we took in stage problem $\mu=1.2\pi^{2}$ with maximum of $10\%$ random disturbance on each step during simulation. All other parameters are the same as before. We can observe on the Subfigures (A),  (B) and  (C) that even subdomain control found by SAC gives us the exponential stabilization rate. These results are promising for future work towards boundary control. On Subfigure (C) it is possible to see that control obtained by SAC drive the system to some sort of equilibrium interval in which the error remains after reaching a certain small enough value. This shows, that the system is robust with respect to small disturbances which is promising in order to use SAC in the direction of robust optimization.

4.2.3. Partial observation.

Along with control on subdomain we also look on the results of control with partial observation. This means that we will change weight constant $\bar{q}$ for matrix $Q$ in the cost function into $q^{*}=\chi_{(a,b)}(x)\bar{q}$, for some given interval $(a,b)$ where we carry observations. The results for different intervals of observations are given in Figure 4. On the Subfigures (A),  (B) and  (C) we can observe, that control found by SAC with partial observation remain the exponential stabilization rate shown before. On Subfigure (C) we can see, that the bigger region of observation will result in better stabilization.

4.2.4. Comparison with LQR method.

Finally we compare SAC with the standard linear-quadratic regulator (LQR) design for finding optimal control solution u. For the case of subdomain control with some disturbance (as described before), the result is given in Figure 3, Subfigure (C). It is possible to see, that in the case of SAC the error is reaching acceptable error faster, then the one using LQR. Furthermore, another big improvement is time of work. While standard LQR took 20.03s to find a solution, SAC finds a solution in just 0.33s, which is a great improvement and a prominent direction towards real-time decision-making. Similar results are obtained when comparing LQR and SAC for full-domain control. Even though LQR drive the solution closer to the desired one (error could be much smaller), SAC drives the solution towards acceptable error faster and took significantly less time to perform.

5. Concluding remarks

Our investigations show that sequential action control (SAC) is a promising framework for control and stabilization of PDE-dynamical problems. It is well suited to include measurements in an online fashion for example in order to account for uncertainties. As a particular variant of a moving horizon method and in contrast to classical model predictive control, the control synthesis can be done without solving a dynamic optimization problem. This makes it very easy to implement the controller. Moreover, from this we can see that the control principle can be easily extended to piecewise linear switched systems and hence arbitrarily close approximations of nonlinear evolutions.

Here we have taken a first step towards a qualitative analysis of this control principle in a Hilbert space framework with distributed control. As in the case of problems with ordinary differential equations, the stability analysis for the important case of quadratic stage costs turns out to be closely related to LQR-theory. We have been applying this prototypically for stabilization of a reaction-diffusion process.

Possible directions for future work is the stability analysis, including decay rates, for problems including boundary control, potentially using the results obtained in Emirsjlow and Townley (2000), partial state observation, other versions of (3), hyperbolic dynamical systems as well as complying with state constraints.

Funding

This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) [Project-ID - 239904186] – TRR 154 - subproject A03.