Taylor Expansions of the Value Function Associated with a Bilinear Optimal Control Problem

TL;DR

This paper develops polynomial Taylor expansions of the value function for a bilinear infinite-dimensional optimal control problem, enabling the design of suboptimal feedback laws with proven approximation properties and an application to Fokker-Planck equations.

Contribution

It introduces a novel method for approximating the value function using multilinear forms derived from generalized Lyapunov equations, advancing control strategies for complex systems.

Findings

Polynomial approximations effectively describe the value function.

The derived feedback law provides near-optimal control performance.

Application demonstrates the method's effectiveness on Fokker-Planck equations.

Abstract

A general bilinear optimal control problem subject to an infinite-dimensional state equation is considered. Polynomial approximations of the associated value function are derived around the steady state by repeated formal differentiation of the Hamilton-Jacobi-Bellman equation. The terms of the approximations are described by multilinear forms, which can be obtained as solutions to generalized Lyapunov equations with recursively defined right-hand sides. They form the basis for defining a suboptimal feedback law. The approximation properties of this feedback law are investigated. An application to the optimal control of a Fokker-Planck equation is also provided.

Equations

$$\underset{u\in L^2(0,\infty )}{\inf }$$

$$\displaystyle\text{where: }\left\{\begin{array}[]{l}\frac{\,\text{d}}{\,\text{d}t}y(t)=Ay(t)+(Ny(t)+B)u(t),\quad\text{for }t>0\\ y(0)=y_{0}.\end{array}\right.$$

$${\mathcal{V}}_p(y):=\sum _{k=2}^p\frac{1}{k!}{\mathcal{T}}_k(y,...,y)$$

$$\mathcal{V}(y)-{\mathcal{V}}_p(y)=\mathcal{O}(\Vert y{\Vert }_Y^{p+1}).$$

$$\sum _{i=1}^k{\mathcal{T}}_k(z_1,...,z_{i-1},A_{\Pi }{z}_i,z_{i+1},...,z_k)={\mathcal{R}}_k(z_1,...,z_k),$$

$$\mathcal{T}(A_{\Pi }{z}_1,z_2)+\mathcal{T}(z_1,A_{\Pi }{z}_2)=\mathcal{R}(z_1,z_2).$$

$${\mathbf{u}}_p(y)=-\frac{1}{\alpha }D{\mathcal{V}}_p(y)(Ny+B)$$

$$\frac{\text{d}}{\text{d}t}y(t)=Ay(t)+(Ny(t)+B)\mathbf{u}(y(t)),y(0)=y_0.$$

$$\mathcal{J}({\mathbf{U}}_p(y_0),y_0)\le \mathcal{V}(y_0)+\mathcal{O}(\Vert y_0{\Vert }_Y^{p+1}).$$

$$\|\mathbf{U}_{p}(y_{0})-\bar{u}\|_{L^{2}(0,\infty)}=\mathcal{O}\big{(}\|y_{0}\|_{Y}^{(p+1)/2}\big{)}.$$

$$a(v,v)\ge \nu \Vert v{\Vert }_V^2-\lambda \Vert v{\Vert }_Y^2\text{for all}v\in V.$$

$$N\in \mathcal{L}(V,Y)\cap \mathcal{L}(\mathcal{D}(A),V)\text{and}{N}^{\ast }\in \mathcal{L}(V,Y),$$

$$\underset{u\in L^2(0,\infty )}{\inf }\mathcal{J}(u,y_0):=\frac{1}{2}{\int }_0^{\infty }\Vert S(u,y_0;t){\Vert }_Y^2\text{d}t+\frac{\alpha }{2}{\int }_0^{\infty }u(t{)}^2\text{d}t,$$

$$\left\{\begin{array}[]{l}\frac{\,\text{d}}{\,\text{d}t}y(t)=Ay(t)+Ny(t)u(t)+Bu(t),\quad\text{for}\;t>0,\\[6.45831pt] y(0)=y_{0}.\end{array}\right.$$

$$W(0,T)=\left\{y\in L^2(0,T;V):\frac{\text{d}}{\text{d}t}y\in L^2(0,T;V^{\ast })\right\}.$$

$$\mathcal{V}(y_0)=\underset{u\in L^2(0,\infty )}{\inf }\mathcal{J}(u,y_0).$$

$$\Vert y{\Vert }_{W(0,T)}\le c(T,\Vert y_0{\Vert }_Y,\Vert u{\Vert }_{L^2(0,T)}).$$

$$\displaystyle\|y\|_{L^{\infty}(0,T;Y)}^{2}\leq\$$

$$\displaystyle\|\tilde{y}-y\|_{L^{\infty}(0,T;Y)}^{2}\leq\$$

$$\displaystyle\|y\|_{L^{\infty}(0,\infty;Y)}^{2}\leq\$$

$$\displaystyle\|y\|_{L^{2}(0,\infty;V)}^{2}\leq\$$

$$\displaystyle\left\|\frac{\mathrm{d}y}{\mathrm{d}t}\right\|_{L^{2}(0,\infty;V^{*})}^{2}\leq\$$

$$[\mathcal{D}(-A_0),Y{]}_{\frac{1}{2}}=[\mathcal{D}(-A_0^{\ast }),Y{]}_{\frac{1}{2}}=V.$$

$$\Vert y{\Vert }_{H^1(0,T;Y)\cap L^2(0,T;\mathcal{D}(-A_0))}\le c(T,\Vert y_0{\Vert }_V,\Vert u{\Vert }_{L^2(0,T)}).$$

$$\left\{\begin{array}[]{l}\frac{\,\text{d}}{\,\text{d}t}z(t)=Az(t)+\tilde{N}y(t)u(t)+\tilde{B}u(t),\quad\text{for}\;t>0,\\[6.45831pt] z(0)=(-A_{0})^{\frac{1}{2}}y_{0},\end{array}\right.$$

$$\Vert (y_1,...,y_k){\Vert }_{Y^k}=\underset{i=1,...,k}{\max }\Vert y_i{\Vert }_Y.$$

$$\Vert \mathcal{T}\Vert :=\underset{y\in B_{Y^k}(1)}{\sup }|\mathcal{T}(y)|<\infty .$$

$$|\mathcal{T}(z_1,...,z_k)|\le \Vert \mathcal{T}\Vert \prod _{i=1}^k\Vert z_i{\Vert }_Y.$$

$$\mathcal{T}(z_{\sigma (1)},...,z_{\sigma (k)})=\mathcal{T}(z_1,...,z_k).$$

$$({\mathcal{T}}_1\otimes {\mathcal{T}}_2)(y_1,\dots ,y_{k+\ell })={\mathcal{T}}_1(y_1,...,y_k){\mathcal{T}}_2(y_{k+1},...,y_{k+\ell }).$$

$$y^{\otimes k}=(y,...,y)\in Y^k.$$

$$|\mathcal{T}(y)-\mathcal{T}(v)|\le k{M}^{k-1}\Vert T\Vert \Vert y-v{\Vert }_{Y^k}.$$

Full text

Keywords: Value function, Hamilton-Jacobi-Bellman equation, bilinear control systems, Riccati equation, generalized Lyapunov equations, Fokker-Planck equation.

AMS Classification: 49J20, 49N35, 93D05, 93D15.

1 Introduction

In this article, we consider the following bilinear optimal control problem:

[TABLE]

Here, $V\subset Y\subset V^{*}$ is a Gelfand triple of real Hilbert spaces, $y_{0}\in Y$, $A\colon\mathcal{D}(A)\subset Y\rightarrow Y$ is the infinitesimal generator of an analytic $C_{0}$-semigroup $e^{At}$ on $Y$, $N\in\mathcal{L}(V,Y)$, $B\in Y$, and $\alpha>0$. Additional assumptions on the system, in particular a stabilizability assumption, will be made in subsections 2.1 and 3.2. The goal pursued with problem (1) is the stabilization of the dynamical system (4) around the steady state 0 when a perturbation $y_{0}$ is applied. We denote by $\mathcal{V}$ the associated value function: for $y_{0}\in Y$, $\mathcal{V}(y_{0})$ is the value of problem (1) with initial condition $y_{0}$.

Rather than investigating this problem as a mathematical programming problem, which associates an optimal open-loop control with a given initial value $y_{0}$, we take the perspective of designing an optimal feedback law. The design of an optimal feedback law is intimately related to the computation of the value function $\mathcal{V}$, which is in general a very difficult task, since $y$ takes values in an infinite-dimensional space. Even after discretization, the computation time needed for obtaining $\mathcal{V}$ usually increases exponentially with the dimension of the discretized state space, a phenomenon known as the curse of dimensionality. Nonetheless, the computation of a feedback law, rather than an open-loop control, is particularly relevant in the context of stabilization problems.

The goal of this article is to construct a Taylor approximation of the value function at the origin, and to derive from this approximation a feedback law which generates good open-loop controls for small values of $y_{0}$. We begin by proving the existence of a sequence of multilinear forms $\mathcal{T}_{k}\colon Y^{k}\rightarrow\mathbb{R}$ such that for any $p\geq 2$,

[TABLE]

is a polynomial approximation of order $p+1$ of the value function $\mathcal{V}$ in the neighborhood of [math], that is to say

[TABLE]

The sequence $(\mathcal{T}_{k})_{k\geq 2}$ is constructed by induction. The bilinear mapping $\mathcal{T}_{2}$ is the solution to an algebraic operator Riccati equation. For all $k\geq 3$, the mapping $\mathcal{T}_{k}$ is the solution to the following generalized Lyapunov equation: for all $z_{1},...,z_{k}\in\mathcal{D}(A)$,

[TABLE]

where the operator $A_{\Pi}$ generates an exponentially stable semigroup on $Y$ and the right-hand side $\mathcal{R}_{k}$ is known and depends on $N$, $B$, $\mathcal{T}_{2}$,…,$\mathcal{T}_{k-1}$ in an explicit fashion. The terminology generalized Lyapunov equations is motivated by the fact that (6) can be seen as a generalization of operator Lyapunov equations, which can typically be written as follows:

[TABLE]

To achieve this task and to present the resulting expressions in an convenient manner, we exploit the symmetry structure of the formal derivatives of $\mathcal{V}$. From the approximation $\mathcal{V}_{p}$ of the value function $\mathcal{V}$, we derive the following feedback law:

[TABLE]

and analyse the associated closed-loop system:

[TABLE]

We denote by $\mathbf{U}_{p}(y_{0})$ the open-loop generated by $\mathbf{u}_{p}$ for a given initial condition $y_{0}$, that is to say, $\mathbf{U}_{p}(y_{0};t)=\mathbf{u}_{p}(y(t))$, where $y(t)$ is the solution to (7). On top of (5), we prove that

[TABLE]

In other words, we prove that the open-loop controls generated by $\mathbf{u}_{p}$ are $\mathcal{O}(\|y_{0}\|^{p+1})$-optimal. We also prove for all $y_{0}$ sufficiently small, there exists an optimal control $\bar{u}$ such that

[TABLE]

In the finite-dimensional case, expansion techniques for Lyapunov functions or for the value function associated with nonlinear control problems have a long history, which dates back at least to [2]. To the best of our knowledge, our article is the first one dealing with Taylor expansions of any order for infinite-dimensional systems. A sophisticated analysis is also required for proving the well-posedness of the closed-loop system associated with $\mathbf{u}_{p}$. Moreover, the convergence rate analysis has apparently received little attention so far, especially concerning the rate of convergence of the suboptimal controls to the optimal ones. As far as we know, estimates (8) and (9) are new. In this respect, we are only aware of the analysis done in [8] for systems of the form: $\frac{\text{d}}{\text{dt}}y(t)=Ay(t)+\varepsilon\varphi(y(t))+Bu(t)$.

Let us mention some additional related literature. In [2], the author considers a general stabilization problem for a nonlinear system that can be expanded in a power series around the origin. It is shown that the optimal control can be characterized in terms of a convergent power series as well. In [14], the expandability of the optimal control for nonlinear analytic and differentiable systems is analyzed in detail. As in the other works on this topic, an important assumption is the local stabilizability of the underlying system. Moreover, it is shown in [14] that the lowest order terms of the approximation are defined by the linearized dynamics. For nonlinear systems with linear controls, in [8, 9], the degree of approximation of the truncated Taylor series to the optimal control is analyzed. In [6], the formal power series approach is discussed for the particular case of bilinear control systems. The explicit structure of the terms up to the third order are given and shown to be unique for locally stabilizable systems. More recent developments, which are based on Taylor series expansions and their use for a numerical approximation of the value function can be found in [15] and [1], as well as in the survey article [11]. For a further detailed overview on obtaining optimal feedback controls including numerical experiments in the finite-dimensional case, we refer to the survey [3] and to the references therein.

In the infinite-dimensional case, we are only aware of the results from [20], where a third-order approximation for a stabilization problem of the Burgers equation with the control entering linearly is investigated theoretically and numerically. To the best of our knowledge, a more general analysis of Taylor approximations for infinite-dimensional control systems does not exist yet.

Our article is structured as follows. Section 3 is a preparatory section. We show that if $\mathcal{V}$ is Fréchet differentiable, then it is the solution to some HJB equation. In Theorem 12, we further show that if $\mathcal{V}$ is $(p+1)$-times differentiable in the neighborhood of 0, then $D^{p}\mathcal{V}(0)$ is a solution to a generalized Lyapunov equation. This result motivates the construction of $\mathcal{V}_{p}$. Our main contributions start in section 4. In this section, we rigorously define the sequence of multilinear forms $(\mathcal{T}_{k})_{k\geq 2}$, the polynomial approximations $\mathcal{V}_{p}$, and the feedback laws $\mathbf{u}_{p}$. In section 5, we prove the well-posedness of the closed-loop system associated with $\mathbf{u}_{p}$, in the neighborhood of [math]. In section 6, we prove the existence of an optimal (open-loop) control and investigate some of its regularity properties. Section 7 contains our main results: in Theorem 30, we prove the error estimates (5) and (8). Estimate (9) is proved in Theorem 32.

2 Analytical preliminaries

2.1 State equation

Throughout the article, $V\subset Y\subset V^{*}$ denotes a Gelfand triple of real Hilbert spaces, where the embedding of $V$ into $Y$ is dense and compact and where $V^{*}$ stands for the topological dual of $V$. Further, $a\colon V\times V\rightarrow\mathbb{R}$ denotes a bounded $V$-$Y$ bilinear form on $V\times V$, i.e. there exist $\nu>0$ and $\lambda\in\mathbb{R}$, such that

[TABLE]

Associated with $a$, there exists a unique closed linear operator $A$ in $Y$ characterized by $\mathcal{D}(A)=\{v\in V\colon w\mapsto a(v,w)\,\text{is}\,Y\text{-continuous}\}$ and by $\langle Av,w\rangle_{Y}=-a(v,w)$, for all $v\in\mathcal{D}(A)$ and $w\in V$, see e.g. [4, Part II, Chapter 1, Section 2.7]. Moreover, $A$ has a uniquely defined extension as bounded linear operator in $\mathcal{L}(V,V^{*})$, which will be denoted by the same symbol, see. [19, Section 2.2]. Further, we choose $N\in\mathcal{L}(V,Y)$, and thus $N^{*}\in\mathcal{L}(Y,V^{*})$. We assume that the restrictions of $N$ and $N^{*}$ to $\mathcal{D}(A)$ and to $V$, together with $N$ satisfy

[TABLE]

where $\mathcal{D}(A)$ is endowed with the graph norm. Moreover, we assume that $B\in Y$ and we choose $\alpha>0$. The inner product on $Y$ is denoted by $\langle\cdot,\cdot\rangle$ or $\langle\cdot,\cdot\rangle_{Y}$ and duality between $V$ and $V^{*}$ by $\langle\cdot,\cdot\rangle_{V,V^{*}}$. We are now prepared to state the problem under consideration:

[TABLE]

where $S(u,y_{0};\cdot)$ is the solution to

[TABLE]

Here, $S(u,y_{0})$ is referred to as solution of (10) if for each $T>0$, it lies in the space

[TABLE]

We recall that $W(0,T)$ is continuously embedded in $C([0,T],Y)$ [13, Theorem 3.1]. Let us note that the origin is a steady state of the uncontrolled system (10). Associated with ($P$) and (10), we define the value function on $Y$:

[TABLE]

The following lemma summarizes some properties of equation (10). The proof is quite standard and therefore deferred to the Appendix.

Lemma 1**.**

Assume that (A1) and (A2) hold. For all $u\in L^{2}(0,\infty)$ and for all $y_{0}\in Y$, there exists a unique solution $y$ to (10) and a continuous function $c$ such that

[TABLE]

Moreover, there exists a constant $C>0$ such that for all $T\geq 0$, for all $u\in L^{2}(0,\infty)$ and for all $y_{0}$ and $\tilde{y}_{0}\in Y$, we have

[TABLE]

If further $y$ lies in $L^{2}(0,\infty;Y)$, the constant $C$ is such that

[TABLE]

Additionally, $\lim_{T\to\infty}\|y(T)\|_{Y}=0$.

Proposition 2**.**

If problem ($P$) admits a feasible control (i.e. a control $u\in L^{2}(0,\infty)$ such that $\mathcal{J}(u,y_{0})<\infty$), then it has a solution.

The proof uses standard arguments and it is therefore given in the Appendix. Note that in Section 5, we construct a feedback law generating feasible controls (for small values of $\|y_{0}\|_{Y}$).

Remark 3**.**

We recall some additional properties of the operator $A$ generated by $a$. First, it is well known that $A$ generates an analytic semigroup, see e.g. [19, Sections 3.6 and 5.4], that we denote by $e^{At}$. Let us set $A_{0}=A-\lambda I$, if $\lambda>0$ and $A_{0}=A$ otherwise. Then $-A_{0}$ has a bounded inverse in $Y$, see [19, page 75], and in particular it is maximal accretive, see [19, 20]. We have $\mathcal{D}(A_{0})=\mathcal{D}(A)$ and the fractional powers of $-A_{0}$ are well-defined. In particular, $\mathcal{D}((-A_{0})^{\frac{1}{2}})=[\mathcal{D}(-A_{0}),Y]_{\frac{1}{2}}:=(\mathcal{D}(-A_{0}),Y)_{2,\frac{1}{2}}$ the real interpolation space with indices 2 and $\frac{1}{2}$, see [4, Proposition 6.1, Part II, Chapter 1].**

For the following regularity result, we require that

[TABLE]

Lemma 4**.**

Let (A1)-(A3) hold. Then, there exists a continuous function $c$ such that for all $T>0$, for all $y_{0}\in V$, and for all $u\in L^{2}(0,T;Y)$ the solution to (10) satisfies $y\in H^{1}(0,T;Y)\cap L^{2}(0,T;\mathcal{D}(-A_{0}))$ and the following estimate holds:

[TABLE]

Proof.

Let $y$ denote the solution to (10) and define $z=(-A_{0})^{\frac{1}{2}}y$. Then, $z$ satisfies

[TABLE]

where $\tilde{N}=(-A_{0})^{\frac{1}{2}}N(-A_{0})^{-\frac{1}{2}}$ and $\tilde{B}=(-A_{0})^{\frac{1}{2}}B$. Since $(-A_{0})^{-\frac{1}{2}}\in{\mathcal{L}}(V,\mathcal{D}(-A_{0}))$ and $(-A_{0})^{\frac{1}{2}}\in{\mathcal{L}}(V,Y)$, we have $\tilde{N}\in{\mathcal{L}}(V,Y)$ and $\tilde{B}\in V^{*}$, where we use (A3). Now we can apply (11) of Lemma 1 to obtain that $z\in H^{1}(0,T;V^{*})\cap L^{2}(0,T;V)$ and this implies that $y\in H^{1}(0,T;Y)\cap L^{2}(0,T;\mathcal{D}(-A_{0}))$ and that (17) holds. ∎

Remark 5**.**

For finite dimensional systems with $V=Y=\mathbb{R}^{n}$, assumptions (A1), (A2), and (A3) are trivially satisfied. In Section 8, we describe an infinite-dimensional control problem associated with a Fokker-Planck equation for which the general assumptions are satisfied.**

2.2 Notation for multilinear forms and differentiability properties

We denote by $B_{Y}(\delta)$ the closed ball of $Y$ with radius $\delta$ and center 0. For $k\geq 1$, we make use of the following norm:

[TABLE]

We denote by $B_{Y^{k}}(\delta)$ the closed ball of $Y^{k}$ with radius $\delta$ and center 0, for the norm $\|\cdot\|_{Y^{k}}$. For $k\geq 1$, we say that $\mathcal{T}\colon Y^{k}\rightarrow\mathbb{R}$ is a bounded multilinear form if for all $i\in\{1,...,k\}$ and for all $z_{1}$,…,$z_{i-1}$,$z_{i+1}$,…,$z_{k}\in Y^{k-1}$, the mapping $z\in Y\mapsto\mathcal{T}(z_{1},...,z_{i-1},z,z_{i+1},...,z_{k})$ is linear and

[TABLE]

We denote by $\mathcal{M}(Y^{k},\mathbb{R})$ the set of bounded multilinear forms. For all $\mathcal{T}\in\mathcal{M}(Y^{k},\mathbb{R})$ and for all $(z_{1},...,z_{k})\in Y^{k}$,

[TABLE]

Bounded multilinear forms $\mathcal{T}\in\mathcal{M}(Y^{k},\mathbb{R})$ are said to be symmetric if for all $z_{1}$,…,$z_{k}\in Y^{k}$ and for all permutations $\sigma$ of $\{1,...,k\}$,

[TABLE]

Given two multilinear forms $\mathcal{T}_{1}\in\mathcal{M}(Y^{k},\mathbb{R})$ and $\mathcal{T}_{2}\in\mathcal{M}(Y^{\ell},R)$, we denote by $\mathcal{T}_{1}\otimes\mathcal{T}_{2}$ the bounded multilinear mapping which is defined for all $(y_{1},...,y_{k+\ell})\in Y^{k+\ell}$ by

[TABLE]

For $y\in Y$, we denote

[TABLE]

Lemma 6**.**

Let $\mathcal{T}\colon Y^{k}\rightarrow\mathbb{R}$ be a multilinear form. Then, $\mathcal{T}\in\mathcal{M}(Y^{k},\mathbb{R})$ if and only if it is continuous. In this case, it is also Lipschitz continuous on bounded subsets of $Y^{k}$. More precisely, for all $M>0$, for all $y$ and $v\in B_{Y^{k}}(M)$,

[TABLE]

The proof is given in the Appendix.

Lemma 7**.**

Let $\mathcal{T}\in\mathcal{M}(Y^{k},\mathbb{R}).$ Then, it is also infinitely many times differentiable. In particular, for all $y=(y_{1},...,y_{k})\in Y^{k}$ and $z=(z_{1},...,z_{k})\in Y^{k}$,

[TABLE]

Moreover, for all $M>0$, for all $y$ and $\tilde{y}\in B_{Y^{k}}(M)$,

[TABLE]

Proof.

The Fréchet differentiability of $\mathcal{T}\in\mathcal{M}(Y^{k},\mathbb{R}),$ as well as formula (23) follow from (22), taking $v_{1}=y_{1}+\theta z_{1}$,…,$v_{k}=y_{k}+\theta z_{k}$. Formula (24) follows directly from formula (23). Formula (25) follows from Lemma 6, from (23), and from the following relation:

[TABLE]

Finally, one can prove by induction that $\mathcal{T}$ is infinitely many times differentiable, observing that $D\mathcal{T}(y_{1},...,y_{k})(z_{1},...,z_{k})$ can be written as a sum of bounded multilinear forms. ∎

The following lemma provides a useful chain rule.

Lemma 8**.**

Let $f\in W^{1,1}(0,\infty;Y^{k})$ and $\mathcal{T}\in\mathcal{M}(Y^{k},\mathbb{R}).$ Then, $F:=\mathcal{T}\circ f$ lies in $W^{1,1}(0,\infty)$ and satisfies

[TABLE]

Proof.

Using the continuous embedding of $W^{1,1}(0,\infty;Y^{k})$ in $L^{\infty}(0,\infty;Y^{k})$, we first obtain that

[TABLE]

Therefore, $F\in L^{1}(0,\infty)$ and $D\mathcal{T}\circ f(\cdot)f^{\prime}(\cdot)\in L^{1}(0,\infty)$. It remains to prove that $D\mathcal{T}\circ f(\cdot)f^{\prime}(\cdot)$ is the derivative of $F$ in the sense of distributions.

Let $(f_{n})_{n\in\mathbb{N}}$ be a sequence in $C^{1}(0,\infty;Y^{k})$, with limit $f$ in $W^{1,1}(0,\infty;Y^{k})$. Let $\phi\in C_{c}^{\infty}(0,\infty)$ be a test function. By the chain rule, we have

[TABLE]

Using the continuous embedding of $W^{1,1}(0,\infty;Y^{k})$ in $L^{\infty}(0,\infty;Y^{k})$, we obtain that $(f_{n})_{n\in\mathbb{N}}$ is bounded in $L^{\infty}(0,\infty;Y^{k})$. Let $M>0$ be an upper bound of $\|f_{n}\|_{L^{\infty}(0,\infty;Y^{k})}$ and $\|f\|_{L^{\infty}(0,\infty;Y^{k})}$. By Lemma 6,

[TABLE]

By Lemma 7,

[TABLE]

Passing to the limit in (26), we obtain that

[TABLE]

which justifies that $F$ is differentiable in the sense of distributions, with $F^{\prime}(\cdot)=D\mathcal{T}\circ f(\cdot)f^{\prime}(\cdot)$. This concludes the proof. ∎

3 Derivation of a generalized Lyapunov equation

The goal of this section is to prove that the derivatives of $\mathcal{V}$ at 0 of order three and more, provided that they exist, are solution to a linear equation, that we call generalized Lyapunov equation. The existence of a unique solution to this equation and its use for approximating $\mathcal{V}$ and designing feedback laws will be discussed in the following sections. Rather than postulating this equation, we derive it from the HJB equation under the assumption that $\mathcal{V}$ is $(k+1)$-times Fréchet differentiable in $Y$, with $k\geq 3$, and under a continuity assumption for optimal controls. We stress that the assumptions on $\mathcal{V}$, in particular the differentiability at 0, are only used to obtain the generalized Lyapunov equation. The results obtained in the following sections do not rely on this assumption.

3.1 Derivation of the HJB equation

We prove in this subsection that the value function $\mathcal{V}$ is a solution the Hamilton-Jacobi-Bellman equation (HJB), under the assumption that $\mathcal{V}$ is continuously differentiable and under a continuity assumption for optimal controls.

Following standard arguments, it can be verified that the dynamic programming principle for the infinite horizon problem holds: for all $y_{0}\in Y$, for all $\tau>0$,

[TABLE]

where $\ell(y,u)=\frac{1}{2}\|y\|_{Y}^{2}+\alpha u^{2}$. Moreover, for $\tau>0$, any control $u\in L^{2}(0,\infty)$ is a solution to problem ($P$) with initial condition $y_{0}$ if and only if $u_{|(0,\tau)}$ minimizes the r.h.s. of (27) and $u_{|(\tau,\infty)}$ is a solution to problem ($P$) with initial condition $S(u,y_{0};\tau)$.

Proposition 9**.**

In addition to (A1)-(A3), assume that there exists an open neighborhood $Y_{0}$ of the origin in $Y$ which is such that the two following statements hold:

For all $y_{0}\in Y_{0}$, problem ($P$) possesses a solution $u$ which is right-continuous at time 0. 2. 2.

The value function is continuously differentiable on $Y_{0}$.

Then, for all $y\in\mathcal{D}(A)\cap Y_{0}$, the following Hamilton-Jacobi-Bellman equation holds:

[TABLE]

Proof.

The proof uses standard arguments. Let $y_{0}\in\mathcal{D}(A)\cap Y_{0}$ be arbitrary. By assumption, there exists an optimal solution $\bar{u}$ to ($P$) with initial condition $y_{0}$ which is right-continuous at time 0. Let $u_{0}$ denote the limit of $\bar{u}$ at time 0. Let $\bar{y}=S(\bar{u},y_{0})$ be the associated state. Our proof is based on the following relations:

[TABLE]

Step 1: proof of (29). By the dynamic programming principle, for all $\tau>0$,

[TABLE]

Thus,

[TABLE]

For any $T>0$, we have $\bar{y}\in C([0,T];Y)$ and therefore, we can fix $\tau_{0}>0$ such that $\bar{y}(\tau)\in Y_{0}$, for all $\tau\in[0,\tau_{0}]$. Relation (29) follows then by passing to the limit in (31), in $Y$, when $\tau\to 0$. By continuity of $\bar{y}$ and $\bar{u}$ at time 0, the first term of the left-hand side of (31) clearly converges to $\ell(y_{0},u_{0})$. To prove the convergence of the second term, we need to prove the differentiability of $\bar{y}$ at time 0 and to establish a chain rule property. For all $\tau\in(0,\tau_{0})$, we have

[TABLE]

The first term of the r.h.s. converges to $Ay_{0}$. Regarding the second one, observe first that by Lemma 4, $\bar{y}\in C([0,\tau_{0}];V)$, therefore, since $\bar{u}$ is right-continuous and $N\in\mathcal{L}(V,Y)$, the function $f\colon s\geq 0\mapsto f(s)\in Y$ is right-continuous at time 0. We have

[TABLE]

Since $A$ generates an analytic semigroup (see Remark 3), there exist $M>0$ and $\omega>0$ such that $\|e^{As}\|_{\mathcal{L}(Y)}\leq Me^{\omega s}$. We obtain

[TABLE]

Moreover, for $\tilde{f}(t):=e^{At}f(0)$, it holds that $\tilde{f}\in C([0,\tau],Y)$, therefore

[TABLE]

Combining (32)-(35), we obtain that

[TABLE]

We now have

[TABLE]

Clearly, the second term of the r.h.s. converges to 0. Using the continuity of $D\mathcal{V}$, and the fact that $(\bar{y}(\tau)-y_{0})/\tau$ is bounded, we obtain

[TABLE]

Passing to the limit in (31), we obtain: $\ell(y_{0},u_{0})+D\mathcal{V}(y_{0})(Ay_{0}+(Ny_{0}+B)u_{0})=0$, which proves (29).

Step 2: proof of (30) and conclusion. Let $u\in\mathbb{R}$ and let $\tilde{u}$ be the piecewise constant control equal to $u$ on $(0,1)$ and equal to 0 on $(1,\infty)$. Let $\tilde{y}=S(y_{0},\tilde{u})$. Then, by (27), for all $\tau\in(0,1)$,

[TABLE]

We can pass to the limit (when $\tau\to 0$) with exactly the same arguments as the ones used in the first part of the proof. We therefore obtain

[TABLE]

Since the l.h.s. in the above expression is equal to 0 for $u=u_{0}$, we deduce that it reaches its minimum 0 at $u=u_{0}$. The l.h.s. being linear-quadratic with respect to $u$, the following relation can easily be obtained:

[TABLE]

Equation (28) follows then from (29) and (36). ∎

3.2 A generalized operator Lyapunov equation

We prove in Theorem 12 that if $\mathcal{V}$ is $(k+1)$-times differentiable, then $D^{k}\mathcal{V}(0)$ is a solution to a generalized Lyapunov equation, by differentiating the HJB equation $k$-times. Note that in this subsection, the $k$-th derivative $D^{k}\mathcal{V}(0)$ is represented by a multilinear form in $\mathcal{M}(Y^{k},\mathbb{R}).$

The case $k=3$.

We assume that $\mathcal{V}$ is four times Fréchet differentiable on $Y$ and that the assumptions of Proposition 9 hold. Note that the differentiability on $Y$ implies the differentiability on $\mathcal{D}(A)$. Differentiating the HJB equation (28) a first time with respect to $y$ in the direction $z_{1}\in\mathcal{D}(A)$ yields

[TABLE]

Differentiating a second time with respect to $y$ in the direction $z_{2}\in\mathcal{D}(A)$, we obtain

[TABLE]

Observing that $\mathcal{V}(y)\geq 0$ for all $y$ and that $\mathcal{V}(0)=0$, we deduce that $D\mathcal{V}(0)=0$. Taking $y=0$ in the above equation and representing $D^{2}\mathcal{V}(0)$ as nonnegative self-adjoint operator $\Pi=\Pi^{*}\in\mathcal{L}(Y)$ such that $D^{2}\mathcal{V}(0)(z_{1},z_{2})=\langle z_{1},\Pi z_{2}\rangle_{Y}$ for all $z_{1},z_{2}\in\mathcal{D}(A)$, we obtain

[TABLE]

Equation (38) is the algebraic operator Riccati equation, see e.g. [7, 12]. Throughout the rest of the paper, we assume, on top of assumptions (A1)-(A3) that

[TABLE]

Since the pair $(A,I)$ is exponentially detectable on $Y$, it follows from [7, Theorem 6.2.7] that (38) has a unique nonnegative stabilizing solution $\Pi\in\mathcal{L}(Y)$. Accordingly, we define the operator $A_{\Pi}$ as follows:

[TABLE]

In particular, since $\Pi$ is stabilizing, we know that the semigroup $e^{A_{\Pi}t}$ is exponentially stable on $Y.$ Moreover, since $BB^{*}\Pi\in\mathcal{L}(Y),$ by a perturbation result for analytic semigroups [16], as in Remark 3 we can choose $\tilde{\lambda}\geq 0$ such that $-\tilde{A}_{0}=-A_{\Pi}+\tilde{\lambda}I$ is maximal accretive. Endowing $\mathcal{D}(-\tilde{A}_{0})$ and $\mathcal{D}(-A_{0})$ with their graph norms, we have that the identity operator between these spaces is a homeomorphism $\mathcal{D}(-\tilde{A}_{0})\cong\mathcal{D}(-A_{0}).$ Consequently, the interpolation spaces defined by the method of traces [4, Part II, Chapter 1, Section 2] are homeomorphic and we thus obtain

[TABLE]

We continue by differentiating a third time with respect to $y$ in the direction $z_{3}\in\mathcal{D}(A),$ which for $y=0$ leads us to:

[TABLE]

We can already observe that this equation is a linear equation with respect to $D^{3}\mathcal{V}(0)$. Moreover, using the symmetry of the derivatives, we can re-write it in the following form:

[TABLE]

where the multilinear form $\mathcal{R}_{3}\colon Y^{3}\rightarrow\mathbb{R}$ is defined by

[TABLE]

Lyapunov equation: general case.

The derivation of the Lyapunov equation, for a general $k\geq 3$, requires some symmetrization techniques for multilinear forms. For $i$ and $j\in\mathbb{N}$, we make use of the following set of permutations:

[TABLE]

where $S_{i+j}$ is the set of permutations of $\{1,...,i+j\}$. A permutation $\sigma\in S_{i,j}$ is uniquely defined by the subset $\{\sigma(1),...,\sigma(i)\}$, therefore, the cardinality of $S_{i,j}$ is equal to the number of subsets of cardinality $i$ of $\{1,...,i+j\}$, that is to say

[TABLE]

Let us give an example. Representing a permutation $\sigma\in S_{4}$ by the vector $(\sigma(1),...,\sigma(4))$, we have:

[TABLE]

Let $\mathcal{T}$ be a multilinear form of order $i+j$. We denote by $\text{Sym}_{i,j}(\mathcal{T})$ the multilinear form defined by

[TABLE]

The two following lemmas contain the main properties related to this specific symmetrization technique which will be needed. Their proofs are given in the Appendix. Lemma 10 is a general Leibnitz formula for the differentiation of the product of two functions. Lemma 11 is a symmetry property.

Lemma 10**.**

Let $f\colon Y\rightarrow\mathbb{R}$ and $g\colon Y\rightarrow\mathbb{R}$ be two $k$-times continuously differentiable functions. Then, for all $k\geq 1$, for all $y\in Y$,

[TABLE]

Lemma 11**.**

Let $\mathcal{T}_{1}\in\mathcal{M}(Y^{i},\mathbb{R})$ and $\mathcal{T}_{2}\in\mathcal{M}(Y^{j},\mathbb{R})$. Then, for all $y\in Y$,

[TABLE]

Moreover, if $\mathcal{T}_{1}$ and $\mathcal{T}_{2}$ are symmetric, then $\text{Sym}_{i,j}(\mathcal{T}_{1}\otimes\mathcal{T}_{2})$ is also symmetric.

We are now ready to derive the generalized Lyapunov equation.

Theorem 12**.**

Let $k\geq 3$. Assume that $\mathcal{V}\colon Y\rightarrow\mathbb{R}$ is $(k+1)$-times Fréchet differentiable in a neighborhood of 0 and that the assumptions of Proposition 9 hold. Then for all $z_{1}$,…,$z_{k}\in\mathcal{D}(A)$,

[TABLE]

where the multilinear form $\mathcal{R}_{k}\colon Y^{k}\rightarrow\mathbb{R}$ is given by:

[TABLE]

Remark 13**.**

The meaning of the expression on the left-hand side of (42) is the following:

[TABLE]

Proof of Theorem 12.

We differentiate the HJB equation $k$ times. First observe that since $k\geq 3$,

[TABLE]

We then have

[TABLE]

Therefore, the $k$-th derivative of $y\mapsto D\mathcal{V}(y)(Ay)$, evaluated at $y=0$, is given by

[TABLE]

For all $y\in\mathcal{D}(A)$ we set $\mathcal{W}(y)=D\mathcal{V}(y)(Ny+B)$. It remains to compute the $k$-th derivative of $y\in\mathcal{D}(A)\mapsto\mathcal{W}(y)^{2}$ at $y=0$. Similarly to (44),

[TABLE]

and therefore,

[TABLE]

Using Lemma 10 and observing that $D^{0}\mathcal{W}(0)=\mathcal{W}(0)=0$, we obtain

[TABLE]

We compute now the summands of the above expression for $i=1$ and $i=k-1$. Note first that

[TABLE]

Therefore,

[TABLE]

and moreover

[TABLE]

Combining (47), (48), and (49), we obtain

[TABLE]

From (43), (45), and (50), we deduce (42). ∎

4 Construction of the polynomial approximation

In this section, we construct a sequence $(\mathcal{T}_{k})_{k\geq 2}$, with $\mathcal{T}_{k}\in\mathcal{M}(Y^{k},\mathbb{R})$, which enables us to obtain a polynomial approximation of the value function $\mathcal{V}$. For all $k\geq 3$, $\mathcal{T}_{k}$ is the unique solution to a multilinear equation, with a right-hand side which depends explicitly on $N$, $B$, and $\mathcal{T}_{2}$,…,$\mathcal{T}_{k-1}$. This multilinear equation is suggested by the structure of (42). The existence will be obtained under the generic assumptions (A1)-(A4).

We start with an existence result for multilinear equations with particular right-hand sides, which will be relevant once we turn to (42).

Theorem 14**.**

Let $k\geq 2$. For $1\leq i<j\leq k,$ let $\mathcal{R}_{ij}\in\mathcal{M}(Y^{k},\mathbb{R}).$ Then, there exists a unique $\mathcal{T}\in\mathcal{M}(Y^{k},\mathbb{R})$ such that for all $(z_{1},...,z_{k})\in\mathcal{D}(A)^{k}$,

[TABLE]

where:

[TABLE]

Moreover, if $\mathcal{R}$ is symmetric, then $\mathcal{T}$ is also symmetric.

Proof.

Part 1: existence. For all $(z_{1},...,z_{k})\in Y^{k}$, we define:

[TABLE]

Let us justify the well-posedness of $\mathcal{T}$. All along the article, the constant $C$ is a generic constant whose value can change. We have

[TABLE]

Here, the last step follows from the fact that $N\in\mathcal{L}(V,Y)$. Using the generalized Hölder inequality, we obtain

[TABLE]

Since the semigroup $e^{A_{\Pi}t}$ is analytic and exponentially stable on $Y$, it follows from [4, Theorem 2.2, Part II, Chapter 3] that

[TABLE]

The same estimate can be derived for the other terms of $\mathcal{R}$. It follows that

[TABLE]

which proves that $\mathcal{T}$ is well-defined on $Y^{k}$. If $\mathcal{R}$ is symmetric, then $\mathcal{T}$ is also symmetric, by (53).

We next prove that $\mathcal{T}$ is a solution to (51). Let us first assume that $(z_{1},...,z_{k})\in\mathcal{D}(A^{2})^{k}$ and define

[TABLE]

We already know that $F\in L^{1}(0,\infty)$, by (53). In fact, $F\in W^{1,1}(0,\infty)$, with

[TABLE]

This is seen as follows. For all $i<j$, for $t\in[0,\infty)$, we define

[TABLE]

so that $F=\sum_{1\leq i<j\leq k}F_{ij}$. To simplify, we focus on $F_{12}$. By [7, Theorem 5.1.5], $A_{\Pi}^{-1}$ exists and $A_{\Pi}^{-1}\in\mathcal{L}(Y,\mathcal{D}(A))$. Using the commutativity of $A_{\Pi}$, $A_{\Pi}^{-1}$, and $e^{A_{\Pi}t}$, we find that

[TABLE]

where

[TABLE]

Since $NA_{\Pi}^{-1}\in\mathcal{L}(Y),$ it follows that $\widehat{\mathcal{R}}_{12}\in\mathcal{M}(Y^{k},\mathbb{R}).$ Moreover, for $z_{i}\in\mathcal{D}(A^{2})$ it holds that $A_{\Pi}e^{A_{\Pi}\cdot}A_{\Pi}z_{i}\in L^{1}(0,\infty;Y)$ and hence, $g_{12}\in W^{1,1}(0,\infty;Y^{k})$. By Lemma 8 we obtain that $F_{12}\in W^{1,1}(0,\infty)$ and that

[TABLE]

Similar formulas can be obtained in the same manner for $F_{ij}$. It follows that $F\in W^{1,1}(0,\infty)$ and that (54) holds. Since $\mathcal{R}$ is continuous and $\|e^{A_{\Pi}t}z_{i}\|_{Y}\underset{t\to\infty}{\longrightarrow}0$, we deduce $F(t)\underset{t\to\infty}{\longrightarrow}0$. Moreover, $F\in W^{1,1}(0,\infty)$ implies that it is absolutely continuous and therefore, for all $T\geq 0$,

[TABLE]

Passing to the limit when $T\to\infty$, we obtain

[TABLE]

Since $F(0)=\mathcal{R}(z_{1},...,z_{k})$, equation (51) is satisfied. Since $\mathcal{D}(A^{2})$ is dense in $\mathcal{D}(A)$, equation (51) remains valid for $z_{i}\in\mathcal{D}(A)$, by continuity.

Part 2: uniqueness. Let $\widetilde{\mathcal{T}}\in\mathcal{M}(Y^{k},\mathbb{R})$ satisfy (51) and let us set $\mathcal{E}=\widetilde{\mathcal{T}}-\mathcal{T}$. For all $(z_{1},...,z_{k})\in\mathcal{D}(A)^{k}$,

[TABLE]

For a fixed $(z_{1},...,z_{k})\in\mathcal{D}(A)^{k}$, we define

[TABLE]

As in the second part of the proof, we can show that $G\in W^{1,1}(0,\infty)$, with

[TABLE]

Note that for all $t$, we have that $e^{A_{\Pi}t}z_{i}\in\mathcal{D}(A).$ Hence, we deduce from (55) that $G^{\prime}(t)=0$ and therefore that $G$ is constant. For all $i$, we have $\|e^{A_{\Pi}t}z_{i}\|_{Y}\underset{t\to\infty}{\longrightarrow}0$, and thus $G(t)\underset{t\to\infty}{\longrightarrow}0$ since $\mathcal{E}$ is continuous. This implies that $G$ is identically 0. Since $G(0)=\mathcal{E}(z_{1},...,z_{k})$ it follows that $\mathcal{E}$ is null on $\mathcal{D}(A)^{k}$. By continuity, $\mathcal{E}$ is null on $Y^{k}$. This concludes the proof. ∎

Remark 15**.**

Theorem 14 can be generalized to equations with a right-hand side of the following form:

[TABLE]

where $\mathcal{R}^{(0)}$, $(\mathcal{R}_{i}^{(1)})_{1\leq i\leq k}$, and $(\mathcal{R}_{ij}^{(2)})_{1\leq i<j\leq k}$ are bounded multilinear forms.

In the following theorem, we use the nonnegative self-adjoint Riccati operator $\Pi$ which was defined in (38).

Theorem 16**.**

There exists a unique sequence of symmetric multilinear forms $(\mathcal{T}_{k})_{k\geq 2,}$ with $\mathcal{T}_{k}\in\mathcal{M}(Y^{k},\mathbb{R})$ and a unique sequence of multilinear forms $(\mathcal{R}_{k})_{k\geq 3},$ with $\mathcal{R}_{k}\in\mathcal{M}(\mathcal{D}(A)^{k},\mathbb{R})$ such that for all $(z_{1},z_{2})\in Y^{2}$,

[TABLE]

and such that for all $k\geq 3$, for all $(z_{1},...,z_{k})\in\mathcal{D}(A)^{k}$,

[TABLE]

Proof.

We prove this claim by induction. The induction assumption is the following: for all $p\geq 2$, there exists a unique family $(\mathcal{T}_{k})_{2\leq k\leq p},$ $\mathcal{T}_{k}\in\mathcal{M}(Y^{k},\mathbb{R})$ and a unique family $(\mathcal{R}_{k})_{3\leq k\leq p},\mathcal{R}_{k}\in\mathcal{M}(\mathcal{D}(A)^{k},\mathbb{R})$ such that (58) and (59) hold, for all $k=3,...,p$.

For $p=2$, it suffices to check that $(z_{1},z_{2})\in Y^{2}\mapsto(z_{1},\Pi z_{2})\in\mathbb{R}$ is continuous, which directly follows from the Cauchy-Schwarz inequality and the fact that $\Pi\in\mathcal{L}(Y)$.

Let $p\geq 2$, assume that the induction assumption is satisfied. Let $(\mathcal{T}_{k})_{2\leq k\leq p},\mathcal{T}_{k}\in\mathcal{M}(Y^{k},\mathbb{R})$ and $(\mathcal{R}_{k})_{3\leq k\leq p},\mathcal{R}\in\mathcal{M}(\mathcal{D}(A)^{k},\mathbb{R})$ be such that (58) and (59) hold, for all $k=3,...,p$.

Let $\mathcal{R}_{p+1}$ be defined by (59b) and (59c) (taking $k=p+1$). The multilinear mapping $\mathcal{R}_{p+1}$ is well-defined, since (59b) and (59c) are defined by $\mathcal{T}_{2}$,…,$\mathcal{T}_{p}$. Moreover, $\mathcal{R}_{p+1}$ can be written as a sum of multilinear mappings in which the operator $N$ appears at most twice. More precisely, since by assumption, $\mathcal{T}_{2}$,…,$\mathcal{T}_{p}$ are bounded, $\mathcal{R}_{p+1}$ can be written in the form (57). Therefore, by Theorem 14, there exists a unique $\mathcal{T}_{p+1}\in\mathcal{M}(Y^{p+1},\mathbb{R})$ satisfying (59a). By induction, $\mathcal{T}_{2}$,…,$\mathcal{T}_{p}$ are all symmetric. One can easily check that for $i=1$,…,$p-2$, $\mathcal{C}_{i}$ is symmetric and for $i=1,...,p-1$, $\mathcal{G}_{i}$ is symmetric. Therefore, by Lemma 11, $\mathcal{R}_{p+1}$ is symmetric and finally, by Theorem 14, $\mathcal{T}_{p+1}$ is symmetric. This proves the induction assumption for $p+1$ and concludes the proof. ∎

Remark 17**.**

In the finite-dimensional case $Y=\mathbb{R}^{n},$ a multilinear form $\mathcal{S}\in\mathcal{M}(Y^{k},\mathbb{R})$ can be naturally identified with a multidimensional array (or tensor) $\mathbf{S}\in\mathbb{R}^{n\times\dots\times n}.$ Denoting with $\mathrm{vec}(\mathbf{S})\in\mathbb{R}^{n^{k}}$ the associated vectorization of $\mathbf{S}$ allows to interpret (51) as a linear tensor equation of the form

[TABLE]

where $I_{n}$ is the identity matrix in $\mathbb{R}^{n\times n}$ and $A_{\Pi,n}\in\mathbb{R}^{n\times n}$ denotes a finite-dimensional approximation of the operator $A_{\Pi}.$ Let us particularly emphasize that these types of equations can often be efficiently solved by tensor methods, see e.g. [10].

For all $p\geq 2$, we define the function $\mathcal{V}_{p}$ as follows:

[TABLE]

where the sequence $(\mathcal{T}_{k})_{k\geq 2}$ is given by Theorem (16). The definition of $\mathcal{V}_{p}$ is motivated by Theorem 12.

Remark 18**.**

In Theorem 30, we prove that $\mathcal{V}_{p}$ is an approximation of order $p+1$ of $\mathcal{V}$, in the neighborhood of 0. This result is obtained without assuming the differentiability of $\mathcal{V}$.**

5 Well-posedness of the closed-loop system

In this section, we analyse the non-linear feedback law $\mathbf{u}_{p}\colon y\in V\rightarrow\mathbb{R}$, defined by

[TABLE]

Its form is suggested by (36) and (60). Note that the explicit expression of $\mathbf{u}_{p}$ follows from Lemma 7 and from the symmetry of the multilinear forms $\mathcal{T}_{k}$. In this section, we discuss the well-posedness of the closed-loop system

[TABLE]

for a fixed value of $p\geq 2$. We recall that throughout this section and the remainder of the paper, assumptions (A1)-(A4) are supposed to hold. In Theorem 22, we will establish the existence of a solution to (62), provided that $\|y_{0}\|_{Y}$ is sufficiently small. We denote this closed-loop solution by

[TABLE]

The distinction with the notation $S(u,y_{0})$ used for an open-loop control $u\in L^{2}(0,\infty;Y)$ will be clear from the context. We also denote by

[TABLE]

the open-loop control generated with the feedback law $\mathbf{u}_{p}$ and the initial condition $y_{0}$. We will prove in Corollary 23 that $\mathbf{U}_{p}(y_{0})$ is well-defined in $L^{2}(0,\infty)$, provided that $\|y_{0}\|_{Y}$ is small enough.

The strategy that we use to prove the well-posedness of (62) is rather standard and has been applied in the context of infinite-dimensional systems several times, see e.g. [5, 17, 20]. It consists in proving that the non-linear part of the closed-loop system satisfies a Lipschitz continuity property. To this purpose, we introduce the nonlinear mapping $F:W_{\infty}\rightarrow L^{2}(0,\infty;V^{*})$ defined by

[TABLE]

It can be expressed as the sum of monomial functions of degree greater or equal to 2. Observe that the closed-loop system (62) can be written as follows:

[TABLE]

In Lemma (20) we prove that $F$ is well-defined and Lipschitz continuous on bounded subsets (for the $L^{\infty}(0,\infty;Y)$-norm), and that the associated Lipschitz modulus can be made as small as necessary, by restricting the size of the considered subset. The well-posedness of (62) is then obtained in Theorem (22) with a fixed-point argument.

We set

[TABLE]

We recall that $W_{\infty}$ is continuously embedded in $C(0,\infty;Y)$ [13, Theorem 3.1]: there exists a constant $C_{0}>0$ such that for all $y\in W_{\infty}$,

[TABLE]

The following lemma is a technical lemma, used for analysing the non-linear mapping $F$.

Lemma 19**.**

There exists a constant $C>0$ such that for all $\delta\in[0,1]$, for all $k=2,...,p$, and for all $y_{1}$ and $y_{2}\in B_{Y}(\delta)\cap V$,

[TABLE]

Proof.

Let $\delta\in[0,1]$, let $y_{1}$ and $y_{2}\in B_{Y}(\delta)$. Then we have

[TABLE]

We need to find a bound on the three terms of the right-hand side of the above inequality. Note first that $\|Ny_{1}+B\|_{V^{*}}\leq M:=\|N\|_{\mathcal{L}(Y,V^{*})}+\|B\|_{Y}$. We have

[TABLE]

For the upper estimate of $(c)$, we used Lemma 6 and the fact that

[TABLE]

The lemma follows, since $\delta^{k-1}\leq\delta$ and since $V$ is continuously embedded in $Y$. ∎

We now prove a Lipschitz continuity property satisfied by $F$.

Lemma 20**.**

The mapping $F$ is well-defined. Moreover, there exists a constant $C_{1}>0$ such that for all $\delta\in[0,1]$, for all $y_{1}$ and $y_{2}\in W_{\infty}$ with $\|y_{1}\|_{L^{\infty}(0,\infty;Y)}\leq\delta$ and $\|y_{2}\|_{L^{\infty}(0,\infty;Y)}\leq\delta$,

[TABLE]

Proof.

Observe that $F(0)=0$. Therefore, (67) will ensure that $F(y)\in L^{2}(0,\infty;V^{*})$ (at least for $\|y\|_{L^{\infty}(0,\infty;Y)}\leq 1$, but the well-posedness can actually be checked for any $y$). Let $y_{1}$ and $y_{2}\in W_{\infty}$ be such that $\|y_{1}\|_{L^{\infty}(0,\infty;Y)}\leq\delta$ and $\|y_{2}\|_{L^{\infty}(0,\infty;Y)}\leq\delta$. By Lemma 19,

[TABLE]

With estimates similar to the ones used in Lemma 19, one can show that

[TABLE]

Using the continuous embedding of $W_{\infty}$ in $L^{\infty}(0,\infty;Y)$, we obtain (67), which concludes the proof. ∎

With regard to a fixed-point argument, let us consider the linearized nonhomogeneous system associated to (62)

[TABLE]

for which we have the following result.

Proposition 21**.**

There exists a constant $C_{2}>0$ such that for all $f\in L^{2}(0,\infty;V^{*})$ and for all $y_{0}\in Y$, there exists a unique mild solution $z\in W_{\infty}$ to (68) satisfying

[TABLE]

In particular, $z\in C_{b}([0,\infty);Y)$.

This result can be verified with the techniques of [4, Theorem 2.2, Part II, Chapter 3] and [20]. We are now ready to prove the well-posedness of (62).

Theorem 22**.**

There exist two constants $\delta_{0}>0$ and $C>0$ such that for all $y_{0}\in B_{Y}(\delta_{0})$, the closed-loop system (62) admits a unique solution $S(\mathbf{u}_{p},y_{0})\in W_{\infty}$ satisfying

[TABLE]

Moreover, the mapping $y_{0}\in B_{Y}(\delta_{0})\mapsto S(\mathbf{u}_{p},y_{0})$ is Lipschitz continuous.

Proof.

In the proof, we denote by $C_{0}$ the constant involved in (66) and by $C_{1}$ and $C_{2}$ the two constants obtained in Lemma 19 and Lemma 20. We set

[TABLE]

Let us fix $y_{0}\in B_{Y}(\delta_{0})$. Consider the mapping $\mathcal{Z}\colon y\in W_{\infty}\mapsto\mathcal{Z}(y)$, where $\mathcal{Z}(y)$ is the solution of

[TABLE]

which exists by Proposition 21. We show that $\mathcal{Z}$ is a contraction in

[TABLE]

Note that $\|y\|_{W_{\infty}}\leq C\|y_{0}\|_{Y}\leq C\delta_{0}$ for all $y\in\Omega$ and that

[TABLE]

Let us show that $\mathcal{Z}(\Omega)\subseteq\Omega$. Let $y\in\Omega$. Applying Lemma 20 (with $\delta=C_{0}C\delta_{0}$), we obtain

[TABLE]

Therefore, by Proposition 21,

[TABLE]

which proves that $\mathcal{Z}(y)\in\Omega$.

Next, for $y_{1}$ and $y_{2}\in\Omega$ we set $z=\mathcal{Z}(y_{2})-\mathcal{Z}(y_{1})$. Then we have

[TABLE]

Taking $\delta=C_{0}C\delta_{0}$ and applying Lemma 20 and Propositon 21, we obtain

[TABLE]

Hence, $\mathcal{Z}$ is a contraction and the well-posedness of (62) follows with the Banach fixed point theorem.

We finally prove that the mapping $y_{0}\in B_{Y}(\delta_{0})\mapsto S(\mathbf{u}_{p},y_{0})$ is Lipschitz continuous. Let $y_{1,0}$ and $y_{2,0}\in B_{Y}(\delta_{0})$, let $y_{1}=S(\mathbf{u}_{p},y_{1,0})$, let $y_{2}=S(\mathbf{u}_{p},y_{2,0})$, let $z=y_{2}-y_{1}$. It holds

[TABLE]

By (70), we obtain

[TABLE]

Applying again Lemma 20 with $\delta=C_{0}C\delta_{0}$, we obtain that

[TABLE]

Therefore, by Proposition 21,

[TABLE]

It follows that

[TABLE]

which proves the Lipschitz continuity of the mapping $y_{0}\mapsto S(\mathbf{u}_{p},y_{0})$ and concludes the proof of the theorem. ∎

Corollary 23**.**

Let $\delta_{0}$ be given by Theorem 22. The following mapping:

[TABLE]

is well-defined and continuous. Moreover, there exists a constant $C>0$ such that for all $y_{0}\in B_{Y}(\delta_{0})$,

[TABLE]

Proof.

We begin by proving the well-posedness and the continuity of $\mathbf{U}_{p}$. We actually prove that the mapping is Lipschitz continuous. Since $\mathbf{U}_{p}(0)=0$, the Lipschitz continuity ensures also the well-posedness. We set $\Omega=S(\mathbf{u}_{p},B_{Y}(\delta_{0}))\subset W_{\infty}$. By Theorem 22, there exists $\delta>0$ such that for all $y\in\Omega$,

[TABLE]

For all $y_{1}$ and $y_{2}\in B_{Y}(\delta)$,

[TABLE]

In the last inequality, we used Lemma 6 and the fact that

[TABLE]

As a consequence, for all $y_{1}$ and $y_{2}\in\Omega$,

[TABLE]

It follows that the mapping: $y\in\Omega\mapsto\mathbf{u}_{p}(y(\cdot))\in W_{\infty}$ is Lipschitz continuous. By composition with $y_{0}\in B_{Y}(\delta_{0})\mapsto S(\mathbf{u}_{p},y_{0})$, the mapping $\mathbf{U}_{p}$ is Lipschitz continuous and well-posed.

Let us prove inequality (71). Since $S(\mathbf{u}_{p},\cdot)$ and $\mathbf{U}_{p}$ are both Lipschitz continuous, there exists $C>0$ such that for all $y_{0}\in B_{Y}(\delta_{0})$,

[TABLE]

It follows that

[TABLE]

which concludes the proof. ∎

6 Properties of the optimal control

Proposition 24**.**

Let $\delta_{0}>0$ be given by Theorem 22. Then, for all $y_{0}\in B_{Y}(\delta_{0})$, there exists a solution $u$ to problem ($P$) with initial value $y_{0}$. Moreover, $y:=S(u,y_{0})$ lies in $L^{2}(0,\infty;V)\cap L^{\infty}(0,\infty;Y)$ and the following estimates hold:

[TABLE]

where the constant $C$ is independent of $y_{0}$.

Proof.

By Corollary 23, we have $\mathcal{V}(y_{0})\leq C\|y_{0}\|_{Y}^{2}\leq C\delta_{0}^{2}$. Hence, Proposition 2 guarantees the existence of a solution $u$ to problem ($P$), with initial condition $y_{0}$. Let $y=S(u,y_{0})$. We deduce from $\mathcal{V}(y_{0})=\|y\|_{L^{2}(0,\infty;Y)}^{2}+\frac{\alpha}{2}\|u\|_{L^{2}(0,\infty)}^{2}$ that

[TABLE]

Estimate (72) follows then from (14), (15), and (73). ∎

Proposition 25**.**

The value function $\mathcal{V}$ is continuous on $B_{Y}(\delta_{0})$, with $\delta_{0}>0$ given by Theorem 22.

Proof.

Let $\varepsilon_{2}>0$. We construct $\varepsilon_{1}>0$ in such a way that for all $\hat{y}_{0}\in B_{Y}(\delta_{0})$ and $\tilde{y}_{0}\in B_{Y}(\delta_{0})$,

[TABLE]

Before defining $\varepsilon_{1}$, we need to introduce some constants. By Corollary 23, there exists a constant $C>0$ such that for all $y_{0}\in B_{Y}(\delta_{0})$, $\mathcal{V}(y_{0})\leq C\|y_{0}\|_{Y}^{2}\leq C\delta_{0}^{2}$. We set

[TABLE]

The constant $T$ is defined in such a way that for each solution $u$ to ($P$) with initial value $y_{0}\in B_{Y}(\delta_{0})$, there exists $\tau\in[0,T]$ such that $\|S(u,y_{0};\tau)\|_{Y}\leq\varepsilon_{3}$. Indeed, if it was not the case, one would have

[TABLE]

in contradiction with Corollary 23. For all $y_{0}\in B_{Y}(\delta_{0})$, it holds: $\mathcal{V}(y_{0})\leq C\delta_{0}^{2}$, and therefore, if $u$ is an optimal solution to ($P$) with initial value $y_{0}$, then

[TABLE]

By Lemma 1, there exist $M$ and $L>0$ such that for all $u\in L^{2}(0,T)$ with $\|u\|_{L^{2}(0,T)}^{2}\leq 2C\delta^{2}/\alpha$, for all $y_{0}$ and $\tilde{y}_{0}\in B_{Y}(\delta_{0})$,

[TABLE]

We finally define

[TABLE]

We are ready to prove (74). Let $\tilde{y}_{0}$ and $\hat{y}_{0}\in B_{Y}(\delta_{0})$ be such that $\|\tilde{y}_{0}-\hat{y}_{0}\|_{Y}\leq\varepsilon_{1}$. Let $\tilde{u}$ and $\hat{u}$ be associated optimal solutions, and let $\tilde{y}$ and $\hat{y}$ be the associated trajectories. Take $\tau\in[0,T]$ such that $\|\tilde{y}(\tau)\|_{Y}\leq\varepsilon_{3}$. By (75) and (76), we have

[TABLE]

We set $y_{1}=S(\tilde{u},\hat{y}_{0};\tau)$. It holds that

[TABLE]

Therefore, using the definition of $\varepsilon_{3}$, we obtain that $\|y_{1}\|_{Y}\leq\delta_{0}$ and thus that

[TABLE]

By the dynamic programming principle, see (27), we have

[TABLE]

We find now an upper estimate on the integral of the r.h.s. in the above inequality. We have

[TABLE]

Combining (78) and (79), we obtain that $\mathcal{V}(\hat{y}_{0})\leq\mathcal{V}(\tilde{y}_{0})+\varepsilon_{2}$. One can similarly prove that $\mathcal{V}(\tilde{y}_{0})\leq\mathcal{V}(\hat{y}_{0})+\varepsilon_{2}$, by exchanging the symbols “\ \hat{}\” and “\ \tilde{}\” in the above proof. Therefore, (74) holds and the continuity of $\mathcal{V}$ is demonstrated. ∎

7 Error estimate for the polynomial approximation

In this section, we prove the two main results of the article. In Theorem 30, we give an estimate for the quality of the feedback law $\mathbf{u}_{p}$ for $\|y_{0}\|_{Y}$ small enough. This will be based on the fact that $\mathcal{V}_{p}$ provides a Taylor approximation of $\mathcal{V}$ of order $p+1$ in a neighborhood in $Y$ of 0. In Theorem 32, we give an estimate for $\|\bar{u}-\mathbf{U}_{p}(y_{0})\|_{L^{2}(0,\infty)}$, where $\bar{u}$ is a solution to problem ($P$) with initial condition $y_{0}$, with $y_{0}$ small enough, and where $\mathbf{U}_{p}(y_{0})$ is the open-loop control associated with the feedback law $\mathbf{u}_{p}$ and the initial condition $y_{0}$ (see the definition given by (63)).

Our analysis consists first in defining a perturbed cost function $\mathcal{J}_{p}$ which has the property that $\mathcal{V}_{p}$ is its minimal value functional over a set of controls specified below. This is achieved by constructing a remainder term $r_{p}$, defined for $p\geq 2$ and $y\in V$ by

[TABLE]

We recall that the definitions of $\mathcal{C}_{i}$ and $\mathcal{G}_{i}$ are given by (59c). The perturbed cost function $\mathcal{J}_{p}$ is defined by

[TABLE]

The well-posedness of $\mathcal{J}_{p}$, for a certain class of controls, will be investigated in Lemma 29. Note that $r_{p}$ is not necessarily non-negative.

Proposition 26**.**

For all $p\geq 2$ and all $y\in\mathcal{D}(A)$, we have

[TABLE]

Moreover, for all $p\geq 2$, there exists a constant $C>0$ such that for all $y\in V$,

[TABLE]

Remark 27**.**

Relation (81) states that $\mathcal{V}_{p}$ is a solution to the HJB equation associated with the problem of minimizing $\mathcal{J}_{p}$ (compare with Proposition (9)). This relation is the key tool to establish Lemma 29. **

Proof of Proposition 26.

Let us prove (81). Let us fix $y\in\mathcal{D}(A)$. For $p=2$, using that the operator $\Pi$ generating $\mathcal{T}_{2}$ is the solution of the algebraic Riccati equation, we obtain

[TABLE]

Now let $p\geq 3$. Our proof is based on Theorem 16. The expressions of the multilinear forms $\mathcal{C}_{i}$, $\mathcal{G}_{i}$, and $\mathcal{R}_{k}$ can be simplified when the mappings are evaluated at $y^{\otimes i}$ and $y^{\otimes k}$, respectively. By definition of $\mathcal{C}_{i}$ and $\mathcal{G}_{i}$ (see (59c)) and using the symmetry of the multilinear forms $\mathcal{T}_{i}$ (proved in Theorem 16),

[TABLE]

Moreover, by definition of $\mathcal{R}_{k}$ (see (59b)) and by Lemma 11, we have

[TABLE]

Using once again the symmetry of the multilinear forms $\mathcal{T}_{k}$, we obtain

[TABLE]

We are now ready to prove (81). We first have

[TABLE]

Moreover, by (83),

[TABLE]

The expression $D\mathcal{V}_{p}(y)(Ny+B)$ is therefore the sum of monomial functions of degree $1$,…,$p$. As a consequence, \big{(}D\mathcal{V}_{p}(y)(Ny+B)\big{)}^{2} can be expressed as a sum of monomial functions $\tilde{q}_{p,2}$, $\tilde{q}_{p,3}$,…,$\tilde{q}_{p,2p}$ of degree $2$,…,$2p$, respectively:

[TABLE]

We compute now these monomial functions. First,

[TABLE]

For $3\leq k\leq p$, we obtain

[TABLE]

The terms $(a)$ and $(b)$ can be expressed explicitly as follows:

[TABLE]

and thus, using (84), relation (88) becomes

[TABLE]

For $p+1\leq k\leq 2p$, we have

[TABLE]

Using (86), (87), (89), and (90), and grouping monomial functions of same degree, we obtain

[TABLE]

The terms in brackets in the above expression are equal to zero by (85). This proves (81).

Let us prove (82). From (80) and Theorem 16, we obtain that for all $p\geq 2$, there exists a constant $\tilde{C}>0$ such that for all $i=1,...,p$, $|q_{p,j}(y)|\leq\tilde{C}\|y\|_{V}\,\|y\|_{Y}^{j-1}$. We deduce that for all $i=p,...,2p$ and all $j=i-p,...,p$,

[TABLE]

Estimate (82) follows then from the definition of $r_{p}$. ∎

Lemma 28**.**

Let $p\geq 2$ and let $\delta_{0}>0$ be the constant given by Theorem 22. Then, there exists a constant $C>0$ such that for all $y_{0}\in B_{Y}(\delta_{0})$,

[TABLE]

where $\bar{y}$ is an optimal trajectory for problem ($P$) with initial value $y_{0}$.

Proof.

By Theorem 22, there exists a constant $C_{1}$ such that for all $y_{0}\in B_{Y}(\delta_{0})$,

[TABLE]

where $y_{p}=S(\mathbf{u}_{p},y_{0})$. By Proposition 24, increasing if necessary the value of $C_{1}>0$, for each solution $\bar{u}$ to problem ($P$) associated to an initial value $y_{0}\in B_{Y}(\delta_{0})$ we have

[TABLE]

where $\bar{y}=S(\bar{u},y_{0})$. Let us denote by $C_{2}$ the constant provided by Proposition 26. We obtain

[TABLE]

and these inequalities also hold for $y_{p}$. The lemma follows with $C=C_{2}\sum_{i=p+1}^{2p}C_{1}^{i}\delta_{0}^{i-(p+1)}$. ∎

In the following lemma, we establish that the control $\mathbf{U}_{p}(y_{0})=\mathbf{u}_{p}(S(\mathbf{u}_{p},y_{0};\cdot))$ obtained from (61) is optimal with respect to $\mathcal{J}_{p}(\cdot,y_{0})$ for small values of $\|y_{0}\|_{Y}$, over all feasible controls for ($P$).

Lemma 29**.**

Let $p\geq 2$ and let $\delta_{0}>0$ be given by Theorem 22. Let ${u}$ be any feasible control for ($P$) with initial value $y_{0}\in B_{Y}(\delta_{0})\cap V$. Then $\mathcal{J}_{p}({u},y_{0})$ and $\mathcal{J}_{p}(\mathbf{U}_{p}(y_{0}),y_{0})$ are finite and

[TABLE]

Proof.

We start with a computation for an arbitrary feasible control associated with an initial condition $y_{0}\in B_{Y}(\delta_{0})\cap V$. There exists at least one such control, namely $\mathbf{U}_{p}(y_{0})$. Let us set ${y}=S({u},y_{0})$. By Lemmas 1 and 4, we have that ${y}\in H^{1}(0,T;Y)$, for every $T>0$. Together with Lemma 28, this implies that $\mathcal{J}_{p}({u},y_{0})$ and $\mathcal{J}_{p}(\mathbf{U}_{p}(y_{0}),y_{0})$ are finite. Moreover, for all $T>0$, we have $y\in W^{1,1}(0,T;Y)$ and by Lemma 8, the chain rule can be applied to each of the bounded multilinear forms which appear as summands in $\mathcal{V}_{p}(y(\cdot))$. Omitting the time variable in what follows, we obtain

[TABLE]

By Proposition 26,

[TABLE]

where

[TABLE]

Hence, it follows that

[TABLE]

We deduce that for an arbitrary feasible $u$,

[TABLE]

We also deduce from (91) that for the specific $u=\mathbf{U}_{p}(y_{0})$,

[TABLE]

since for this control, the squared expression vanishes. By Lemma 1, we have $\lim_{T\to\infty}y(T)=0\text{ and }\lim_{T\to\infty}y_{p}(T)=0\text{ in }Y.$ Together with the continuity of $\mathcal{V}_{p}$ established in Proposition 25, this implies that

[TABLE]

Finally, passing to the limit in (92) and (93), we obtain

[TABLE]

The lemma is proved. ∎

We now prove that $\mathcal{V}_{p}$ is a Taylor expansion of $\mathcal{V}$ and analyse the quality of the feedback law $\mathbf{u}_{p}$ in the neighborhood of 0.

Theorem 30**.**

Let $\delta_{0}>0$ be given by Theorem 22, let $C$ be the constant given by Lemma 29. Then, for all $y_{0}\in B_{Y}(\delta_{0})$,

[TABLE]

Proof.

We first prove the result for $y_{0}\in B_{Y}(\delta_{0})\cap V$. The following inequalities follow directly from Lemma 28 and Lemma 29 and from the suboptimality of $\mathbf{U}(y_{0})$:

[TABLE]

where $\bar{u}$ is a solution to ($P$) with initial value $y_{0}$. Therefore,

[TABLE]

which proves inequalities (94) and (95) for $y_{0}\in B_{Y}(\delta_{0})\cap V$. By Lemma 6, $\mathcal{V}_{p}$ is continuous, by Proposition 25, $\mathcal{V}$ is continuous on $B_{Y}(\delta_{0})$. By Theorem 22 and Corollary 23, the mappings: $y_{0}\in B_{Y}(\delta_{0})\mapsto S(\mathbf{U}_{p}(y_{0}),y_{0})$ and $y_{0}\in B_{Y}(\delta_{0})\mapsto\mathbf{U}_{p}(y_{0})$ are both continuous. Moreover, the following mapping is continuous:

[TABLE]

Therefore, by composition, the mapping $y_{0}\in B_{Y}(\delta_{0})\mapsto\mathcal{J}(\mathbf{U}_{p}(y_{0}),y_{0})$ is continuous. Finally, since $B_{Y}(\delta_{0})\cap V$ is dense in $B_{Y}(\delta_{0})$, we can pass to the limit in inequalities (94) and (95). They are therefore satisfied for all $y_{0}\in B_{Y}(\delta_{0})$. The theorem follows. ∎

Remark 31**.**

Inequality (94) gives an estimate for the approximation quality of the feedback law $\mathbf{u}_{p}$ in the neighborhood of 0. In general, an inequality like (95) does not imply that $\mathcal{V}$ is $p$-times differentiable in the neighborhood of [math]. Indeed, consider the function

[TABLE]

Then, for all $x\in\mathbb{R}$, $|f(x)|\leq|x|^{3}$, however, $f$ is not continuously differentiable at 0, since for all $x\neq 0$, $f^{\prime}(x)=3x^{2}\sin(1/x^{2})-2\cos(1/x^{2})$, thus $f^{\prime}(x)\nrightarrow 0$ when $x\downarrow 0$.**

We finally give an error estimate for the closed-loop control $\mathbf{U}_{p}(y_{0})$ associated with $\mathbf{u}_{p}$, for small values of $y_{0}$.

Theorem 32**.**

Let $\delta_{0}$ be given by Theorem 22. There exist $\delta_{1}\in(0,\delta_{0}]$ and $C>0$ such that for all $y_{0}\in B_{Y}(\delta_{1})$, there exists a solution $\bar{u}$ to problem ($P$) with initial value $y_{0}$ satisfying the following error estimates:

[TABLE]

where $\bar{y}=S(\bar{u},y_{0})$.

Proof.

The value of $\delta_{1}$ is fixed to $\delta_{0}$ for the moment. We first prove the result for $y_{0}\in B_{Y}(\delta_{1})\cap V$, as in the proof of Theorem 30. Let $\bar{u}$ be a solution to problem ($P$) with initial condition $y_{0}$ and let $\bar{y}=S(\bar{u},y_{0})$, $u_{p}=\mathbf{U}_{p}(y_{0})$, $y_{p}=S(\mathbf{u}_{p},y_{0})$. By Theorem 22 and Proposition 24, there exists a constant $C$ independent of $y_{0}$ such that

[TABLE]

Let us emphasize the fact that in the proof, the mapping $\mathbf{u}_{p}(\bar{y}(\cdot))\in L^{2}(0,\infty)$ plays an important role. It can be seen as an “intermediate” control between $\bar{u}$ and $u_{p}$.

Step 1: estimation of $\|\bar{u}(\cdot)-\mathbf{u}_{p}(\bar{y}(\cdot))\|_{L^{2}(0,\infty)}$. Since $y_{0}\in V$, equality (91) holds for $(u,y)=(\bar{u},\bar{y})$ and therefore, for a.e. $t\geq 0$,

[TABLE]

Integrating on $[0,T]$ and passing to the limit when $T\rightarrow\infty$, as in the proof of Lemma 29, we obtain that

[TABLE]

and finally that

[TABLE]

as a consequence of Theorem 30 and Lemma 28.

Step 2: estimation of $\|\bar{y}-y_{p}\|_{W_{\infty}}$. We use in this part of the proof ideas similar to the ones developed for the well-posedness of the closed-loop system in Theorem 22. We make use of the mapping $F$, defined by (64). Remember that this mapping contains the non-linearities of the closed-loop system (see (65)). Let us set

[TABLE]

Omitting the time variable, we have

[TABLE]

We also have

[TABLE]

Setting $z=\bar{y}-y_{p}$, we obtain

[TABLE]

We compute now estimates of $\|F(\bar{y})-F(y_{p})\|_{L^{2}(0,\infty;V^{*})}$ and $\|f\|_{L^{2}(0,\infty;V^{*})}$, in order to obtain an estimate of $\|z\|_{W_{\infty}}$ with Proposition 21. By definition of $f$, we have

[TABLE]

where the last inequality follows from the estimates (97) and (98). Since $\|\bar{y}\|_{L^{\infty}(0,\infty;Y)}\leq C\delta_{1}$ and $\|\bar{y}_{p}\|_{L^{\infty}(0,\infty;Y)}\leq C\delta_{1}$, we obtain with Lemma 20 that

[TABLE]

We can now reduce the value of $\delta_{1}$ to

[TABLE]

By (99), (100), and Proposition 21,

[TABLE]

It follows that

[TABLE]

The first estimate in (96) is now proved.

Step 3: estimation of $\|\bar{u}-u_{p}\|_{L^{2}(0,\infty)}$. Observing that $u_{p}(\cdot)=\mathbf{u}_{p}(y_{p}(\cdot))$, we obtain that

[TABLE]

We obtain an estimate of the last term of the r.h.s. by proving a Lipschitz property for the mapping $y\in W_{\infty}\mapsto\mathbf{u}_{p}(y(\cdot))\in L^{2}(0,\infty)$. With similar estimates to the ones used in the proof of Lemma 19, one can easily show that for $y_{1}$ and $y_{2}\in B_{Y}(C\delta_{1})$, for all $k=2,...,p$,

[TABLE]

By (97), $\|\bar{y}\|_{L^{\infty}(0,\infty;Y)}\leq C\delta_{1}$ and $\|y_{p}\|_{L^{\infty}(0,\infty;Y)}\leq C\delta_{1}$. Therefore,

[TABLE]

Combining this estimate with (101), we obtain the second inequality of (96).

Step 4: general case. Let $y_{0}\in B_{Y}(\delta_{1})$. Take a sequence $(y_{0}^{k})_{k\in\mathbb{N}}$ in $B_{Y}(\delta_{1})\cap V$ converging to $y_{0}$. As we proved in the first three steps of this proof, for all $k\in\mathbb{N}$, there exists a solution $\bar{u}^{k}$ to problem ($P$) with initial condition $y_{0}^{k}$ such that

[TABLE]

where $\bar{y}^{k}=S(\bar{u}^{k},y_{0}^{k})$. Using arguments similar to the ones used in the proof of Proposition 2, we obtain that there exists an accumulation point $(\bar{u},\bar{y})$ to the sequence $(\bar{u}^{k},\bar{y}^{k})$ for the weak topology of $L^{2}(0,\infty)\times W_{\infty}$ which is such that $\bar{u}$ is a solution to problem ($P$) with initial condition $y_{0}$ and such that $\bar{y}=S(\bar{u},y_{0})$. By Corollary 23, the mapping $\mathbf{U}_{p}$ is continuous. Therefore, we can pass to the limit in (102) and finally obtain the estimates

[TABLE]

which concludes the proof. ∎

Remark 33**.**

The constants $\delta_{0}$, $\delta_{1}$, and $C$, which are provided by Theorem 30 and Theorem 32, depend on $p$.

8 Stabilization of a Fokker-Planck equation

In this section, we show that assumptions (A1)-(A4) are satisfied for a concrete infinite-dimensional bilinear optimal control problem. Following the setup discussed in [5], we focus on the controlled Fokker-Planck equation

[TABLE]

where $\tilde{\nu}>0,\;\Omega\subset\mathbb{R}^{n}$ denotes a bounded domain with smooth boundary $\Gamma=\partial\Omega$, and $\rho_{0}$ denotes an initial probability distribution with $\int_{\Omega}\rho_{0}(x)\mathrm{d}x=1.$ To apply the results from [5], we assume that $\alpha$ and $G\in W^{1,\infty}\cap W^{2,\max(2,n)}(\Omega)$, and that the control shape function fulfills $\nabla\alpha\cdot\vec{n}=0$ on $\Gamma.$ We introduce $\rho_{\infty}=\frac{e^{-\Phi}}{\int_{\Omega}e^{-\Phi}\,\text{d}x},$ where $\Phi=\log\tilde{\nu}+\frac{W}{\tilde{\nu}}$, and observe that $\rho_{\infty}$ is an eigenstate associated with the eigenvalue [math]. While the system is known to converge to this stationary distribution, this can happen inadequately slowly and a control mechanism becomes relevant. Considering (103) as an abstract bilinear control, we arrive at

[TABLE]

where the operators $A$ and $N$ are given by

[TABLE]

In order to consider (103) as a stabilization problem of the form ($P$), we introduce a state variable $y:=\rho-\rho_{\infty}$ as the deviation to the stationary distribution. As discussed in [5], this yields a system of the form

[TABLE]

where

[TABLE]

Since $\int_{\Omega}B\,\,\text{d}x=\int_{\Omega}N{\rho_{\infty}}\,\,\text{d}x=0,$ the control does not influence the one-dimensional subspace associated with $\rho_{\infty}.$ Therefore, a splitting of the state space in the form

[TABLE]

by means of the projection $P$ defined by

[TABLE]

was introduced in [5]. We thus focus on

[TABLE]

where

[TABLE]

and $I_{P}\colon Y_{P}\to Y$ denotes the injection of $Y_{P}$ into $Y.$ With system (104), we associate the cost functional

[TABLE]

Let us verify that the assumptions (A1)-(A4) are satisfied with $Y=Y_{P}$ and $V=H^{1}(\Omega)\cap Y_{P}$, endowed with the inner products from $L^{2}(\Omega)$ and $H^{1}(\Omega)$ respectively and for the bilinear system (104) with operators $\widehat{A}$, $\widehat{N}$, and $\widehat{B}$. Concerning (A1), we have for every $v\in V$ that

[TABLE]

Thus (A1) holds with $\nu=\frac{\tilde{\nu}}{2}$ and $\lambda=\frac{1}{2\tilde{\nu}}\|\nabla G\|^{2}_{L^{\infty}(\Omega)}$ Using that $P^{*}y=y-\int_{\Omega}\rho_{\infty}y\,\,\text{d}x\,\mathbbm{1}$, we further obtain that

[TABLE]

since $\nabla\alpha\cdot\vec{n}=0$ and $I_{P}^{*}\psi=\psi-\frac{1}{\Omega}\int_{\Omega}\psi\,dx\;\mathbbm{1}$. It is now clear that (A2) holds. Assumption (A3) is satisfied with $V=H^{1}(\Omega)\cap Y_{P}$, see e.g. [4, Part II, Chapter 1, Section 6]. Finally, the exponential stability of the uncontrolled system (104) (i.e. with $u=0$) implies assumption (A4) with $F=0$, see [5, Section 4].

9 Conclusions

Techniques for the computation of a Taylor expansion of the value function associated with an optimal control problem have been extended to the case of an infinite-dimensional bilinear system. Explicit formulas have been derived for the right-hand side of the generalized Lyapunov equations arising for the terms of order three and more. Non-linear feedback laws have been derived from the Taylor expansions. Their efficiency has been proved theoretically with new error estimates. It is planned to investigate the use of the resulting generalized Lyapunov equations together with model reduction techniques in an independent study. Generalizations of our results in several directions are possible and can be of interest. These include the case of vector-valued controls and more general dynamical systems.

Acknowledgements

This work was partly supported by the ERC advanced grant 668998 (OCLOC) under the EU’s H2020 research program.

Appendix A Proofs

In this Appendix, we provide the proofs for several results which were used in the main part of the manuscript.

Proof of Lemma 1.

The existence can be proved by standard Galerkin arguments and the a-priori estimates below. To verify these estimates and to alleviate the notation, we often omit the time variable $t$. We first prove estimates (11) and (12). Multiplying the state equation by $y$ and using (A1), we obtain

[TABLE]

By Young’s inequality,

[TABLE]

Therefore, combining (106) and (107),

[TABLE]

We integrate (108) (without the term $\nu\|y\|_{V}^{2}$) and apply Gronwall’s inequality: for all $t\in[0,T]$,

[TABLE]

Estimate (12) is proved. Using (108) once again, together with (12) and the state equation (10), estimate (11) follows. Let us prove (13). Let us set $\delta y=S(u,\tilde{y}_{0})-S(u,y_{0})$. We have

[TABLE]

therefore, using the same techniques as for the derivation of (108), we obtain

[TABLE]

and finally, by Gronwall’s inequality,

[TABLE]

Estimate (13) is proved.

We now assume that $y\in L^{2}(0,\infty;Y)$. We integrate estimate (108) and obtain

[TABLE]

Estimate (14) follows with Gronwall’s inequality. From (108), we also obtain

[TABLE]

Estimate (15) follows directly by integration. Finally, for a.e. $t\geq 0$,

[TABLE]

Estimate (16) follows directly by integration.

To verify the asymptotic behavior, we use the fact that $y\in L^{2}(0,T,Y)$ and $y\in C([0,T],Y)$ imply the existence of a monotonically increasing sequence of numbers $(t_{k})_{k=1}^{\infty}$ such that $\|y(t_{k})\|_{Y}\to 0,t_{k}\to\infty$ as $k\to\infty$. Since $y\in W(0,\infty)$ for any $T>0$, we have that

[TABLE]

see [18, Proposition 1.2, Chapter 3]. Given any $T>0$ and choosing $t_{k}>T$, we estimate

[TABLE]

for $t_{k}\to\infty,T\to\infty.$ ∎

Proof of Proposition 2.

Since there exists a feasible control and since $\mathcal{J}$ is bounded from below, $\mathcal{V}(y_{0})$ is finite and there exists a minimizing sequence $(u_{n})_{n\in\mathbb{N}}$ in $L^{2}(0,\infty)$ with associated states $y_{n}:=S(u_{n},y_{0})$. By definition of $\mathcal{J}$, the sequences $(u_{n})_{n\in\mathbb{N}}$ and $(y_{n})_{n\in\mathbb{N}}$ are bounded in $L^{2}(0,\infty)$ and $L^{2}(0,\infty;Y)$, respectively. We deduce from estimates (14), (15), and (16), that the sequence $(y_{n})_{n\in\mathbb{N}}$ is bounded in $W(0,\infty)$. Extracting if necessary a subsequence, there exists $(\bar{u},\bar{y})\in L^{2}(0,\infty)\times W(0,\infty)$ such that $(u_{n},y_{n})\rightharpoonup(\bar{u},\bar{y})$ in $L^{2}(0,\infty)\times W(0,\infty)$.

We prove now that $\bar{y}=S(\bar{u},y_{0})$. Let $T>0$, let $v\in W(0,T)\cap L^{\infty}(0,T;V)$ be arbitrary. For all $n\in\mathbb{N}$, we have

[TABLE]

Since $\frac{\,\text{d}}{\,\text{d}t}y_{n}\rightharpoonup\frac{\,\text{d}}{\,\text{d}t}\bar{y}$ in $L^{2}(0,T;V^{*})$, we can pass to the limit in the l.h.s. of the above equality. Moreover, since $Ay_{n}\rightharpoonup A\bar{y}$ in $L^{2}(0,T;V^{*})$,

[TABLE]

We also have

[TABLE]

The first integral in the r.h.s. of (110) is bounded by

[TABLE]

and therefore converges to 0, since $\|y_{n}-\bar{y}\|_{L^{2}(0,T;Y)}\underset{n\to\infty}{\longrightarrow}0$ by the Aubin-Lions lemma. Since $\bar{y}\in C(0,T;Y)$ and $N^{*}v\in L^{2}(0,T;Y)$, it holds that: $\langle\bar{y}(\cdot),N^{*}v(\cdot)\rangle\in L^{2}(0,T)$ and therefore, the second integral in (110) converges to 0. We can pass to the limit in (109). We thus obtain:

[TABLE]

Since $W(0,T)\cap L^{\infty}(0,T;V)$ is dense in $W(0,T)$, we obtain that $\bar{y}=S(\bar{u},y_{0})$.

Finally, since the following mapping is convex:

[TABLE]

it is also weakly lower semi-continuous and therefore,

[TABLE]

which proves the optimality of $\bar{u}$. ∎

Proof of Lemma 6.

One can easily check that if $\mathcal{T}$ is not bounded, then it is not continuous at [math]. Assume now that $\mathcal{T}$ is bounded. Let $M>0$, let $y=(y_{1},...,y_{k})\in Y^{k}$ and $v=(v_{1},...,v_{k})\in Y^{k}$ be such that $\|y\|_{Y^{k}}\leq M$ and $\|v\|_{Y^{k}}\leq M$. Then, by (21),

[TABLE]

The lemma is proved. ∎

Proof of Lemma 10.

We prove the lemma by induction. The case $k=1$ is trivially satisfied, since $S_{0,1}$ and $S_{1,0}$ both consist of the unique permutation of the set $\{1\}$.

Let $k\geq 1$, let us assume that formula (41) holds. Before proving (41) for $k+1$, we make an important observation on the structure of $S_{i,k+1-i}$, for $i=1,...,k$. For any $\sigma\in S_{i,k+1-i}$, either $\sigma(i)=k+1$ or $\sigma(k+1)=k+1$. More precisely, we can describe $S_{i,k+1-i}$ as follows:

[TABLE]

Let us assume that $f$ and $g$ are $(k+1)$-times differentiable. Let $(z_{1},...,z_{k+1})\in Y^{k+1}$, using the induction assumption and the fact that $|S_{i,k-i}|=\binom{k}{i}$, we obtain

[TABLE]

In the sum involved in term $(a)$, we isolate the value $i=k$. Note that $S_{k,0}$ only contains one permutation, the identity on $\{1,...,k\}$. We also perform a change of index for the remaining values of $i$. We finally obtain for term $(a)$ the following expression:

[TABLE]

Observe that the last term of the r.h.s. can be written as follows:

[TABLE]

Isolating the value $i=0$ in the sum involved in term $(b)$, we obtain

[TABLE]

Observe that the last term of the r.h.s. can be written as follows:

[TABLE]

We can now combine (112)-(117). In particular, the terms involved in the sums in (114) and (116) can be combined together thanks to the representation of $S_{i,k+1-i}$ provided in (112). We finally obtain

[TABLE]

In the last inequality, we used that $|S_{i,k+1-i}|=\binom{k+1}{i}$. The Leibnitz formula is proved for $k+1$. This concludes the proof. ∎

Proof of Lemma 11.

The first part of the lemma follows directly from the definition and from the fact that $|S_{i,j}|=\binom{i+j}{i}$. Assume that $\mathcal{T}_{1}$ and $\mathcal{T}_{2}$ are symmetric. Let us set $f\colon y\in Y\mapsto\mathcal{T}_{1}(y^{\otimes i})$ and $g\colon y\in Y\mapsto\mathcal{T}_{2}(y^{\otimes j})$. By Lemma 7, the functions $f$ and $g$ are both infinitely many times differentiable. Applying the Leibnitz formula to $fg$, we obtain

[TABLE]

The derivatives of $f$ of order $k>i$ are all null and the derivatives of $g$ of order $k>j$ are also all null. Therefore, in the above sum, all the terms vanish, except the one obtained for $\ell=i$. Moreover, since $\mathcal{T}_{1}$ and $\mathcal{T}_{2}$ are symmetric,

[TABLE]

We therefore obtain that

[TABLE]

This proves that \text{Sym}_{i,j}\big{(}\mathcal{T}_{1}\otimes\mathcal{T}_{2}\big{)} is a symmetric multilinear form, since it can be expressed as the $(i+j)$-th derivative of an infinitely many times differentiable function. The lemma is proved. ∎