Integral and measure-turnpike properties for infinite-dimensional optimal control systems

TL;DR

This paper establishes integral and measure-turnpike properties for infinite-dimensional optimal control systems, linking strict dissipativity and strong duality, and providing insights into long-term behavior of optimal trajectories.

Contribution

It introduces the first general integral-turnpike result for infinite-dimensional systems with terminal constraints and connects strict dissipativity with measure-turnpike properties.

Findings

Integral-turnpike property holds around a set for infinite-dimensional control problems.

Strict dissipativity implies measure-turnpike property in constrained systems.

Strong duality ensures strict dissipativity and measure-turnpike properties.

Abstract

We first derive a general integral-turnpike property around a set for infinite-dimensional non-autonomous optimal control problems with any possible terminal state constraints, under some appropriate assumptions. Roughly speaking, the integral-turnpike property means that the time average of the distance from any optimal trajectory to the turnpike set con- verges to zero, as the time horizon tends to infinity. Then, we establish the measure-turnpike property for strictly dissipative optimal control systems, with state and control constraints. The measure-turnpike property, which is slightly stronger than the integral-turnpike property, means that any optimal (state and control) solution remains essentially, along the time frame, close to an optimal solution of an associated static optimal control problem, except along a subset of times that is of small relative Lebesgue measure as the time horizon is large. Next, we prove that strict strong duality, which is a classical notion in optimization, implies strict dissipativity, and measure-turnpike. Finally, we conclude the paper with several comments and open problems.

Equations

$$\boxed{\begin{array}{rl}& \inf \frac{1}{T}{\int }_0^T{f}^0(y(t),u(t))dt,\\ & \text{subject to}\dot{y}(t)=f(y(t),u(t)),t\in [0,T],\end{array}}$$

$$\boxed{\begin{array}{rl}& \inf {\int }_0^1{f}^0(y(s),u(s))ds,\\ & \text{subject to}\epsilon \dot{y}(s)=f(y(s),u(s)),s\in [0,1].\end{array}}$$

$$\begin{gathered} \boxed{\inf f^0(y,u),\text{subject to}f(y,u)=0. \\ } \end{gathered}$$

$$\boxed{(P_{[t_{0},t_{1}]})\qquad\left\{\begin{array}[]{l}J_{[t_{0},t_{1}]}=\inf\frac{1}{t_{1}-t_{0}}\int_{t_{0}}^{t_{1}}f^{0}(t,y(t),u(t))\,dt,\\[5.69054pt] \text{subject to}\;\;\;\;\dot{y}(t)=A(t)y+f(t,y(t),u(t)),\quad t\in[t_{0},t_{1}],\\[5.69054pt] R(t_{0},y(t_{0}),t_{1},y(t_{1}))=0,\quad(y(t),u(t))\in E\times F,\quad t\in[t_{0},t_{1}].\\[5.69054pt] \end{array}\right.}$$

$$y(\tau )=\Phi (\tau ,t_0)y(t_0)+{\int }_{t_0}^{\tau }\Phi (\tau ,t)f(t,y(t),u(t))dt,\forall \tau \in [t_0,t_1].$$

$$C_{[t_0,t_1]}(y(\cdot ),u(\cdot ))={\int }_{t_0}^{t_1}{f}^0(t,y(t),u(t))dt$$

$$J_{[t_0,t_1]}=\inf \left\{\frac{1}{t_1-t_0}{C}_{[t_0,t_1]}(y(\cdot ),u(\cdot ))\mid (y(\cdot ),u(\cdot ))\text{ admissible},R(t_0,y(t_0),t_1,y(t_1))=0\right\}.$$

$$\boxed{(\bar{P}_{[t_{0},t_{1}]})\qquad\left\{\begin{array}[]{l}\bar{J}_{[t_{0},t_{1}]}=\inf\frac{1}{t_{1}-t_{0}}C_{[t_{0},t_{1}]}(y(\cdot),u(\cdot)),\\[5.69054pt] \text{subject to}\;\;\;\;\dot{y}(t)=A(t)y+f(t,y(t),u(t)),\quad t\in[t_{0},t_{1}],\\[5.69054pt] \quad(y(t),u(t))\in E\times F,\quad t\in[t_{0},t_{1}].\\[5.69054pt] \end{array}\right.}$$

$${\bar{J}}_{[t_0,t_1]}=\inf \left\{\frac{1}{t_1-t_0}{C}_{[t_0,t_1]}(y(\cdot ),u(\cdot ))\mid (y(\cdot ),u(\cdot ))\text{ admissible}\right\}.$$

$${\bar{J}}_{[t_0,+\infty )}=\underset{t_1\to +\infty }{\lim }{\bar{J}}_{[t_0,t_1]}.$$

$$V_{[t_0,t_1]}(y)=\inf \left\{\frac{1}{t_1-t_0}{C}_{[t_0,t_1]}(y(\cdot ),u(\cdot ))\mid (y(\cdot ),u(\cdot ))\text{ admissible},y(t_0)=y\right\}.$$

$$V_{[t_0,+\infty )}(y)=\underset{t_1\to +\infty }{\lim }{V}_{[t_0,t_1]}(y).$$

$$\forall t_0<t_1,J_{[t_0,t_1]}\ge {\bar{J}}_{[t_0,t_1]},$$

$$\underset{t_1\to +\infty }{\mathrm{liminf}}{J}_{[t_0,t_1]}\ge {\bar{J}}_{[t_0,+\infty )}.$$

$$\forall t_0<t_1,\forall y\in X,V_{[t_0,t_1]}(y)\ge {\bar{J}}_{[t_0,t_1]},$$

$$\forall y\in X,V_{[t_0,+\infty )}(y)\ge {\bar{J}}_{[t_0,+\infty )}.$$

$${\bar{J}}_{[t_0,t_1]}=\underset{y\in X}{\inf }{V}_{[t_0,t_1]}(y).$$

$${\bar{J}}_{[t_0,t_1]}=\underset{y\in X}{\inf }\underset{\stackrel{(y(\cdot ),u(\cdot ))\text{ admissible}}{y(t_0)=y}}{\inf }\frac{1}{t_1-t_0}{C}_{[t_0,t_1]}(y(\cdot ),u(\cdot ))=\underset{y\in X}{\inf }{V}_{[t_0,t_1]}(y).$$

$$\forall t_0\in \mathbb{R},\forall y\in \mathcal{T},V_{[t_0,+\infty )}(y)={\bar{J}}_{[t_0,+\infty )}.$$

$$V_{[t_0,+\infty )}(y)=\underset{t\to +\infty }{\lim }\frac{1}{t-t_0}{C}_{[t_0,t]}(y(\cdot ),u(\cdot )).$$

$$V_{[t_0,t_1]}(\hat{y})\ge \underset{y\in X}{\inf }{V}_{[t_0,t_1]}(y)+\frac{1}{t_1-t_0}{\int }_{t_0}^{t_1}\beta (\text{dist}(\hat{y}(t),\mathcal{T}))dt+\mathrm{o}(1),$$

$$\;\;\;\;\;\;\;\;\inf\,\frac{1}{t_{1}-t_{0}}\int_{t_{0}}^{t_{1}}\Big{(}\|y(\cdot,t)\|_{L^{2}(\Omega)}^{2}+\|u(\cdot,t)\|_{L^{2}(\mathcal{D})}^{2}\Big{)}\,dt$$

$$\{\begin{array}{rl}& y_t-\Delta y={\chi }_{\mathcal{D}}u,\text{in}\Omega \times (t_0,t_1),\\ & y=0,\text{on}\partial \Omega \times (t_0,t_1),\\ & y(\cdot ,t_0)=y_0,y(\cdot ,t_1)=0,\text{in}\Omega ,\\ & \Vert u(\cdot ,t){\Vert }_{L^2(\mathcal{D})}\le M,\text{for a.e.}t\in (t_0,t_1).\end{array}$$

$$V_{[t_0,t_1]}(y_0)\ge \frac{1}{t_1-t_0}{\int }_{t_0}^{t_1}\Vert y(\cdot ,t){\Vert }_{L^2(\Omega )}^{2}dt.$$

$$\underset{t_1\to +\infty }{\lim }{J}_{[t_0,t_1]}={\bar{J}}_{[t_0,+\infty )}.$$

$$\underset{t_1\to +\infty }{\lim }\frac{1}{t_1-t_0}{\int }_{t_0}^{t_1}\beta (\text{dist}(y(t),\mathcal{T}))dt=0,$$

$$\frac{1}{t_1-t_0}{\int }_{t_0}^{t_1}\beta (\text{dist}(y(t),\mathcal{T}))dt\le \delta$$

$$Q^{\varepsilon}_{[t_{0},t_{1}]}=\big{\{}t\in[t_{0},t_{1}]\ \mid\ \text{dist}(y(t),\mathcal{T})>\varepsilon\big{\}},\;\;\;\;\forall t_{1}\geq T_{0}.$$

$$\frac{\left|Q_{[t_0,t_1]}^{\epsilon }\right|}{t_1-t_0}\le \frac{\delta }{\beta (\epsilon )},\forall t_1\ge T_0.$$

$$\begin{gathered} \frac{1}{t_1-t_0}{C}_{[t_0,t_1]}(\tilde{y}(\cdot ),\tilde{u}(\cdot ))=\frac{1}{t_1-t_0}{C}_{[t_0,t_0+{\delta }_0]}(\tilde{y}(\cdot ),\tilde{u}(\cdot ))+\frac{1}{t_1-t_0}{C}_{[t_1-{\delta }_1,t_1]}(\tilde{y}(\cdot ),\tilde{u}(\cdot )) \\ +\frac{1}{t_1-t_0}{C}_{[t_0+{\delta }_0,t_1-{\delta }_1]}(\tilde{y}(\cdot ),\tilde{u}(\cdot )). \end{gathered}$$

Full text

††footnotetext: Email address: emmanuel.trelat@upmc.fr (Emmanuel Trélat), zhangcansx@163.com (Can Zhang)

Keywords. Measure-turnpike, strict dissipativity, strong duality, state and control constraints.

AMS subject classifications. 49J20, 49K20, 93D20.

1 Introduction

We start this paper with an intuitive idea in general terms. Consider the optimal control problem

[TABLE]

under some terminal state conditions, with $T>0$ large. Setting $s=t/T$ and $\varepsilon=1/T$, we rewrite the above optimal control problem as

[TABLE]

Then, we expect that, as $\varepsilon\rightarrow 0$, there is some convergence to the static problem

[TABLE]

This intuition has been turned into rigorous results in the literature, under some appropriate assumptions. These results say roughly that, if $T$ is large, then any optimal solution $y(\cdot)$ on $[0,T]$ spends most of its time close to an optimal solution $y_{s}$ of the static problem. This is the (neighborhood) turnpike phenomenon. We call the point $y_{s}$ a turnpike point.

This turnpike phenomenon was first observed and investigated by economists for discrete-time optimal control problems (see, e.g., [11, 20]). In the last three decades, many turnpike results have been established in a large number of works (see, e.g., [1, 2, 6, 7, 9, 12, 16, 19, 25, 32, 33, 34] and references therein), either for discrete-time or continuous-time problems involving control systems in finite-dimensional state spaces, and very few of them in the infinite dimensional setting.

A more quantitative turnpike property, which is called the exponential turnpike property, has been established in [22, 23, 29] for both the linear and nonlinear continuous-time optimal controlled systems. It means that the optimal solution for the dynamic controlled problem remains exponentially close to an optimal solution for the corresponding static controlled problem within a sufficiently large time interval contained in the long-time horizon under consideration. We stress that in those works not only the optimal state and control, but also the corresponding adjoint vector, resulting from the application of the Pontryagin maximum principle, were shown to remain exponentially close to an extremal triple for a corresponding static optimal control problem, except at the extremities of the time horizon. The main ingredient in the papers [22, 23, 29] is an exponential dichotomy transformation and the hyperbolicity feature of the Hamiltonian system, deriving from the Pontryagin maximum principle, under some controllability and observability assumptions.

However, not all turnpike phenomena are around a single point. For instance, the turnpike theorem for calculus of variations problems in [25] is proved for the case when there are several turnpikes. More precisely, they show that there exists a competition between the several turnpikes for optimal trajectories with different initial states, and provide in particular a criterion for the choice of turnpikes that are in competition. On the another hand, for some classes of optimal control problems for periodic systems, the turnpike phenomenon may occur around a periodic trajectory, which is itself characterized as being the optimal solution of an appropriate periodic optimal control problem (cf., e.g., [26, 28, 33, 34, 35]).

In this paper, the first main result is to derive a more general turnpike result, valid for very general classes of optimal control problems settled in an infinite-dimensional state space, and where the turnpike phenomenon is around a set $\mathcal{T}$. This generalizes the standard case where $\mathcal{T}$ is a singleton, and the less standard case where $\mathcal{T}$ is a periodic trajectory. Between the case of one singleton and the periodic trajectory, however, there are, to our knowledge, very few examples for intermediate situations in the literature.

The organization of the paper is as follows. In Section 2, we build up an abstract framework to derive a general turnpike phenomenon around a set. In Section 3, we enlighten the relationship between the above-mentioned abstract framework and the strict dissipativity property. Under the strict dissipativity assumption for optimal control problems, we establish the so-called measure-turnpike property. In Section 4, we provide some material to clarify the relationship between measure-turnpike, strict dissipativity and strong duality. Finally, Section 5 concludes the paper.

2 An abstract setting

In this section, we are going to derive a general turnpike phenomenon around a set $\mathcal{T}$. The framework is the following.

Let $X$ (resp., $U$) be a reflexive Banach space endowed with the norm $\|\cdot\|_{X}$ (resp., $\|\cdot\|_{U}$). Let $f:\mathbb{R}\times X\times U\rightarrow X$ be a continuous mapping that is uniformly Lipschitz continuous in $(y,u)$ for all $t\in\mathbb{R}$. Let $f^{0}:\mathbb{R}\times X\times U\rightarrow\mathbb{R}$ be a continuous function that is bounded from below. Let $E$ and $F$ be two subsets of $X$ and $U$, respectively. Given any $t_{0}\in\mathbb{R}$ and $t_{1}\in\mathbb{R}$ with $t_{0}<t_{1}$, we consider the non-autonomous optimal control problem

[TABLE]

Here, $(A(t),D(A(t)))$ is a family of unbounded operators on $X$ such that the existence of the corresponding two-parameter evolution system $\Phi(t,s)$ is ensured (cf., e.g., [21, Chapter 5, Definition 5.3]), the controls are Lebesgue measurable functions $u(\cdot):[t_{0},t_{1}]\rightarrow F$, and $Y$ is a Banach space, the mapping $R:\mathbb{R}\times X\times\mathbb{R}\times X\rightarrow Y$ stands for any possible terminal state conditions. Throughout the paper, the solutions $(y(\cdot),u(\cdot))\in C([t_{0},t_{1}];X)\times L^{2}(t_{0},t_{1};U)$ are considered in the mild sense, meaning that

[TABLE]

Remark 1**.**

Typical examples of terminal conditions are the following:*

• •

When both initial and final conditions are let free in $(P_{[t_{0},t_{1}]})$, take $R=0$.

• •

When the initial point is fixed (i.e., $y(t_{0})=y_{0}$) and the final point is let free, take $R(s_{0},z_{0},s_{1},z_{1})=z_{0}-y_{0}$.

• •

When both initial and final conditions are fixed (i.e., $y(t_{0})=y_{0}$ and $y(t_{1})=y_{1}$), take $R(s_{0},z_{0},s_{1},z_{1})=(z_{0}-y_{0},z_{1}-y_{1})$.

• •

When the final point is expected to coincide with the initial point (i.e., $y(t_{0})=y(t_{1})$ without any other constraint), for instance in a periodic optimal control problem, in which one assumes that there exists $T>0$ such that $f(t+T,y,u)=f(t,y,u)$ and $f^{0}(t+T,y,u)=f^{0}(t,y,u)$, $\forall(t,y,u)\in\mathbb{R}\times X\times U$, take $R(s_{0},z_{0},s_{1},z_{1})=(s_{1}-s_{0}-T,z_{0}-z_{1})$.

Hereafter, we call $(y(t),u(t))$, $t\in[t_{0},t_{1}]$, an admissible pair if it verifies the state equation and the constraint $(y(t),u(t))\in E\times F$ for almost every $t\in[t_{0},t_{1}]$. We remark that the definition of admissible pair does not require that the terminal state condition $R(t_{0},y(t_{0}),t_{1},y(t_{1}))=0$ is satisfied. We denote by

[TABLE]

the cost of an admissible pair $(y(\cdot),u(\cdot))$ on $[t_{0},t_{1}]$. In other words, $J_{[t_{0},t_{1}]}$ is the infimum with time average cost (Cesàro mean) over all admissible pairs satisfying the constraint on terminal points:

[TABLE]

Throughout the paper, we assume that the problem $(P_{[t_{0},t_{1}]})$ has optimal solutions, and that an admissible pair $(y(\cdot),u(\cdot))$, with initial state $y(t_{0})$, is said to be optimal for the problem $(P_{[t_{0},t_{1}]})$ if $R(t_{0},y(t_{0}),t_{1},y(t_{1}))=0$ and $\frac{1}{t_{1}-t_{0}}C_{[t_{0},t_{1}]}(y(\cdot),u(\cdot))=J_{[t_{0},t_{1}]}$. Existence of optimal solutions for optimal control problems is well-known under appropriate convexity assumptions on $f^{0}$, $f$ and $R$ with $E$ and $F$ convex and closed (see, for instance, [18, Chapter 3]).

We then consider the optimal control problem

[TABLE]

Compared with the problem $(P_{[t_{0},t_{1}]})$, in the above problem there is no terminal state constraint, i.e., $R(\cdot)=0$. In fact, it is the infimum with time average cost over all possible admissible pairs:

[TABLE]

We say the problem $(\bar{P}_{[t_{0},t_{1}]})$ has a limit value if $\lim_{t_{1}\rightarrow+\infty}\bar{J}_{[t_{0},t_{1}]}$ exists. We refer [15, 24] for the sufficient conditions ensuring the existence of the limit value. More precisely, asymptotic properties of optimal values, as $t_{1}$ tends to infinity, have been studied in [24] under suitable nonexpansivity assumptions, and in [15, Corollary 4 (iii)] by using occupational measures. In the sequel, we assume it exists and is written as

[TABLE]

Besides, given any $y\in X$ we define the value function

[TABLE]

It is the optimal value of the optimal control problem with fixed initial data $y(t_{0})=y$ (but free final point). Note that, if there exists no admissible trajectory starting at $y$ (because $E$ would not contain $y$), then we set $V_{[t_{0},t_{1}]}(y)=+\infty$. For each $y\in X$, we say a limit value exists if $\lim_{t_{1}\rightarrow+\infty}V_{[t_{0},t_{1}]}(y)$ exists. We now assume that, for each $y\in X$, the limit value exists and is written as

[TABLE]

Clearly, we have

[TABLE]

and thus

[TABLE]

Meanwhile,

[TABLE]

and thus

[TABLE]

Remark 2**.**

If the optimal control problem is autonomous (i.e., $A(\cdot)=A$, $f$ and $f^{0}$ are independent of time variable), it follows from the definitions that $\bar{J}_{[t_{0},+\infty)}$, as well as $V_{[t_{0},+\infty)}(y)$, $\forall y\in X$, do not depend on $t_{0}\in\mathbb{R}$.

Remark 3**.**

Actually we have

[TABLE]

This is obvious because we can split the infimum and write

[TABLE]

In order to state the general turnpike result, we make the following assumptions:

• $(H_{1})$.

(Turnpike set) There exists a closed set $\mathcal{T}\subset X$ (called turnpike set) such that

[TABLE]

• $(H_{2})$.

(Viability) The turnpike set $\mathcal{T}$ is viable, meaning that, for every $y\in\mathcal{T}$ and for every $t_{0}\in\mathbb{R}$, there exists an admissible pair $(y(\cdot),u(\cdot))$ such that $y(t_{0})=y$ and $y(t)\in\mathcal{T}$ for every $t\geq t_{0}$. Moreover, every admissible trajectory remaining in $\mathcal{T}$ is optimal in the following sense: for every $y\in\mathcal{T}$, for every $t_{0}\in\mathbb{R}$, for every admissible pair $(y(\cdot),u(\cdot))$ such that $y(t_{0})=y$ and $y(t)\in\mathcal{T}$ for every $t\geq t_{0}$, we have

[TABLE]

• $(H_{3})$.

(Controllability) There exist $\bar{\delta}_{0}>0$ and $\bar{\delta}_{1}>0$ such that, for every $t_{0}\in\mathbb{R}$ and every $t_{1}\in\mathbb{R}$ with $t_{1}>t_{0}+\bar{\delta}_{0}+\bar{\delta}_{1}$, and every optimal trajectory $y(\cdot)$ for the problem $(P_{[t_{0},t_{1}]})$,

• –

there exist $\delta_{0}\in(0,\bar{\delta}_{0}]$ and an admissible pair $(y_{0}(\cdot),u_{0}(\cdot))$ on $[t_{0},t_{0}+\delta_{0}]$ such that $y_{0}(t_{0})=y(t_{0})$ and $y_{0}(t_{0}+\delta_{0})\in\mathcal{T}$,

• –

for every $y\in\mathcal{T}$, there exist $\delta_{1}\in(0,\bar{\delta}_{1}]$ and an admissible pair $(y_{1}(\cdot),u_{1}(\cdot))$ on $[t_{1}-\delta_{1},t_{1}]$ such that $y_{1}(t_{1}-\delta_{1})=y$ and $y_{1}(t_{1})=y(t_{1})$.

• $(H_{4})$.

(Coercivity) There exist a monotone increasing continuous function $\beta:[0,+\infty)\rightarrow[0,+\infty)$ with $\beta(0)=0$ and a distance $\text{dist}(\cdot,\mathcal{T})$ to $\mathcal{T}$ such that for every $t_{0}$ and every $\hat{y}\in X$,

[TABLE]

holds for any optimal trajectory $\hat{y}(\cdot)$ starting at $\hat{y}(t_{0})=\hat{y}$ for the problem $(P_{[t_{0},t_{1}]})$, where the last term in the above inequality is an infinitesimal quantity as $t_{1}\rightarrow+\infty$.

Hereafter, we speak of Assumption (H) in order to designate assumptions $(H_{1})$, $(H_{2})$, $(H_{3})$ and $(H_{4})$.

Remark 4**.**

• (i).

*Under $(H_{1})$, we actually have $\bar{J}_{[t_{0},+\infty)}=\inf_{y\in X}V_{[t_{0},+\infty)}(y)$, $\forall t_{0}\in\mathbb{R}$. *

• (ii).

$(H_{2})$ means that, starting at $y\in\mathcal{T}$, it is better to remain in $\mathcal{T}$ than to leave this set.

• (iii).

*$(H_{3})$ is a specific controllability assumption. For instance, in the case that the initial point $y(t_{0})=y_{0}$ and the final point $y(t_{1})=y_{1}$ in the problem $(P_{[t_{0},t_{1}]})$ are fixed, then $(H_{3})$ means that the turnpike set $\mathcal{T}$ is reachable from $y_{0}$ within time $\bar{\delta}_{0}$, and that $y_{1}$ is reachable from any point of $\mathcal{T}$ within time $\bar{\delta}_{1}$. When the turnpike set $\mathcal{T}$ is a single point, we refer the reader to [12] for a similar assumption. *

• (iv).

*$(H_{4})$ is a coercivity assumption involving the value function and the turnpike set $\mathcal{T}$. It may not be easy to verify this condition. However, under the strict dissipativity property (which will be introduced in the next section), it is satisfied. We refer the reader to Section 3 for more discussions about the relationship with the strict dissipativity. *

We first give a simple example which satisfies the Assumption (H).

Example 1**.**

Let $\Omega\subset\mathbb{R}^{n}$, $n\geq 1$, be a bounded domain with a smooth boundary $\partial\Omega$, and let $\mathcal{D}\subset\Omega$ be a non-empty open subset. We denote by $\chi_{\mathcal{D}}$ the characteristic function of $\mathcal{D}$. Let $M>0$ and $y_{0}\in L^{2}(\Omega)$ be arbitrarily given. For $t_{0}<t_{1}$, consider the following optimal control problem for the heat equation:

[TABLE]

subject to

[TABLE]

Here, we take $X=L^{2}(\Omega)$, $U=L^{2}(\mathcal{D})$ and $F=\{u\in U\,\,|\,\,\|u\|_{L^{2}(\mathcal{D})}\leq M\}$. By the standard energy estimate, we can take $E=\{y\in X\,\,|\,\,\|y\|_{L^{2}(\Omega)}\leq\|y_{0}\|_{L^{2}(\Omega)}+M/\lambda_{1}\}$, where $\lambda_{1}>0$ is the first eigenvalue of the Laplace operator with zero Dirichlet boundary condition on $\partial\Omega$.

It is clear that $\bar{J}_{[t_{0},+\infty)}=0$. Let us define the turnpike set $\mathcal{T}=\{0\}$. By the $L^{\infty}$-null controllability and exponential decay of the energy of heat equations, then the above control system with bounded controls is null controllable from each given point $y_{0}$ within a large time interval (see, e.g., [30]). Therefore, the assumptions $(H_{1})$, $(H_{2})$ and $(H_{3})$ are satisfied. Let $y(\cdot)$ be any optimal trajectory starting at $y(\cdot,t_{0})=y_{0}$. By the definition of value function, we see that

[TABLE]

Hence $(H_{4})$ is satisfied with $\beta(r)=r^{2}$, $r\geq 0$.

The main result of this paper is the following. It says that a general turnpike behavior occurs around the turnpike set $\mathcal{T}$, in terms of the time average of the distance from optimal trajectories to $\mathcal{T}$.

Theorem 1**.**

Assume that $f^{0}$ is bounded on $\mathbb{R}\times E\times F$.

1. (i).

Under $(H_{1})$, $(H_{2})$ and $(H_{3})$, for every $t_{0}\in\mathbb{R}$ we have

[TABLE] 2. (ii).

Further, under the additional assumption $(H_{4})$ we have

[TABLE]

for any $t_{0}$ and any optimal trajectory $y(\cdot)$ of the problem $(P_{[t_{0},t_{1}]})$.

Remark 5**.**

*The boundedness assumption on $f^{0}$ in Theorem 1 can be removed in case the optimal control problem is autonomous, i.e., when $A$, $f$ and $f^{0}$ do not depend on $t$, provided that the controllability assumption $(H_{3})$ be slightly reinforced, by assuming “controllability with finite cost”: one can steer $y(t_{0})$ to the turnpike set $\mathcal{T}$ within time $\delta_{0}$ and steer any point of $\mathcal{T}$ to $y(t_{1})$ within time $\delta_{1}$ with a cost that is uniformly bounded with respect to every optimal trajectory $y(\cdot)$ and $y\in\mathcal{T}$. For non-autonomous control problems, see also Remark 7. *

The property (2.3) is a weak turnpike property, which can be called the $\beta$-integral-turnpike property, and which is even weaker than the measure-turnpike property introduced further in Section 3.2. Indeed, from $\eqref{guai1}$ we infer that for any $\delta>0$, there exists $T_{0}>t_{0}$ such that

[TABLE]

for any $t_{1}\geq T_{0}$. If, for any $\varepsilon>0$, we set

[TABLE]

Throughout the paper, we denote by $|Q|$ the Lebesgue measure of a subset $Q\subset\mathbb{R}$. Then, by Markov’s inequality, one can easily derive that

[TABLE]

This is weaker than the property (3.4) in Section 3.2.

Proof of Theorem 1.

$(i)$. Let $t_{1}>t_{0}+\bar{\delta}_{0}+\bar{\delta}_{1}$, with $\bar{\delta}_{0}$ and $\bar{\delta}_{1}$ as in $(H_{3})$. Let $(y(\cdot),u(\cdot))$ be an optimal pair for the problem $(P_{[t_{0},t_{1}]})$. By $(H_{2})$ and $(H_{3})$, there exist $\delta_{0}\in(0,\bar{\delta}_{0}]$, $\delta_{1}\in(0,\bar{\delta}_{1}]$ and an admissible pair $(\widetilde{y}(\cdot),\widetilde{u}(\cdot))$ such that

• •

$\widetilde{y}(\cdot)$ steers the control system from $y(t_{0})$ to $\mathcal{T}$ within the time interval $[t_{0},t_{0}+\delta_{0}]$,

• •

$\widetilde{y}(\cdot)$ remains in $\mathcal{T}$ within the time interval $[t_{0}+\delta_{0},t_{1}-\delta_{1}]$,

• •

$\widetilde{y}(\cdot)$ steers the control system from $\widetilde{y}(t_{1}-\delta_{1})\in\mathcal{T}$ to $y(t_{1})$ within the time interval $[t_{1}-\delta_{1},t_{1}]$.

These trajectories are drawn in Figure 1.

Its cost of time average within the time interval $[t_{0},t_{1}]$ is

[TABLE]

Since $f^{0}$ is bounded on $\mathbb{R}\times E\times F$, the first two terms on the right hand side of (2.4) converge to zero as $t_{1}\rightarrow+\infty$. Since $\widetilde{y}(t_{0}+\delta_{0})\in\mathcal{T}$, by $(H_{2})$ we have

[TABLE]

As $\widetilde{y}(t_{0}+\delta_{0})\in\mathcal{T}$, by $(H_{1})$ we infer

[TABLE]

We now claim that

[TABLE]

We postpone the proof of this claim and first see how it could be used in showing the convergence (2.2). Therefore, we derive from (2.6) and (2.7) that

[TABLE]

This, together with (2.4) and (2.5), indicate that

[TABLE]

On the other hand, by the construction above, $(\widetilde{y}(\cdot),\widetilde{u}(\cdot))$ is an admissible pair satisfying the terminal state constraint $R(t_{0},\widetilde{y}(t_{0}),t_{1},\widetilde{y}(t_{1}))=0$, we have

[TABLE]

This, combined with (2.8), infers that

[TABLE]

Which, along with (2.1), leads to (2.2).

Next, we present the proof of the claim (2.7). Let $(\bar{y}(\cdot),\bar{u}(\cdot))$ be an optimal pair for the problem $(\bar{P}_{[t_{0},t_{1}]})$. Then

[TABLE]

Since $(\bar{y}(\cdot),\bar{u}(\cdot))$ is also admissible for the problem $(\bar{P}_{[t_{0}+\delta_{0},t_{1}]})$,

[TABLE]

we see that

[TABLE]

By the boundedness of $f^{0}$ on $\mathbb{R}\times E\times F$ (i.e., there exists $M>0$ such that $|f^{0}(\cdot)|\leq M$), we obtain

[TABLE]

which implies (2.7) as $t_{1}\rightarrow+\infty$.

$(ii)$. By the definition of the value function $V_{[t_{0},t_{1}]}(\cdot)$, we obtain

[TABLE]

By $(H_{4})$ and Remark 3, we have

[TABLE]

as $t_{1}\rightarrow+\infty$. By (2.2) we infer

[TABLE]

This, together with (2.9) and (2.10), indicates

[TABLE]

which completes the proof. ∎

Remark 6**.**

In the proof of Theorem 1, the role of controllability assumption $(H_{3})$ is to ensure that there is an admissible trajectory $\widetilde{y}(\cdot)$ satisfying the terminal state condition $R(t_{0},\widetilde{y}(t_{0}),t_{1},\widetilde{y}(t_{1}))=0$ and with a comparable cost (i.e., (2.8)).

Note that $(H_{3})$ can be weakened to some cases where controllability may fail: take any control system that is asymptotically controllable to the turnpike set $\mathcal{T}$. This is the case for the heat equation which is asymptotically controllable for any given point (cf., e.g., [18, Chapter 7]). Then, if one waits for a certain time, one will arrive at some neighborhood of $\mathcal{T}$. Similarly, to run the proof as in Theorem 1, one needs an assumption which is stronger than $(H_{2})$. More precisely, one needs viability, not only along $\mathcal{T}$, but also in a neighborhood of $\mathcal{T}$. Under these assumptions, we believe that one can design a turnpike result for this control system with free final point.

In any case, note that, when the final point is free, having a turnpike property is more or less equivalent to having an asymptotic stabilization to $\mathcal{T}$ (see also an analogous discussion in [12, Remark 2]). If additionally one wants to fix the final point, then one would need the existence of a trajectory steering any point of the neighborhood of $\mathcal{T}$ to the final point.

Remark 7**.**

*As seen in the proof of Theorem 1, the assumption of boundedness of $f^{0}$ is used two times: the first one, in order to bound the first two terms of (2.4); the second one, in order to prove (2.7). For autonomous optimal control problems, on the one part we have $\bar{J}_{[t_{0}+\delta_{0},+\infty)}=\bar{J}_{[t_{0},+\infty)}$ (see Remark 2) and then (2.7) is true, and on the other part the first two terms at the right-hand side of (2.4) converge to zero as $t_{1}\rightarrow+\infty$ under the “controllability with finite cost” assumption mentioned in Remark 5. In contrast, for non-autonomous optimal control problems the situation may be more complicated, in particular due to the dependence on time of $f^{0}$. The assumption of boundedness of $f^{0}$ is quite strong and could of course be weakened in a number of ways so as to ensure that the above proof still works. We prefer keeping this rather strong assumption in order to put light in the main line of the argument, not going into too technical details. Variants are easy to derive according to the context. *

3 Relationship with (strict) dissipativity

In this section, we make precisely the relationship between the strict dissipativity property (which we recall in Section 3.1) and the so-called measure-turnpike property (which we define in Section 3.2).

3.1 What is (strict) dissipativity

To fix ideas, in this section we only consider the autonomous case. Let $X$, $U$, $E$ and $F$ be the same as in Section 2. Let $A(\cdot)\equiv A$ generate a $C_{0}$ semigroup $\{e^{tA}:t\geq 0\}$ on $X$, and let $f$ and $f^{0}$ be time-independent. To simplify the notation, for every $T>0$, we here consider the optimal control problem

[TABLE]

Indeed, the above problem $(\bar{P}_{[0,T]})$ coincides with $(\bar{P}_{[t_{0},t_{1}]})$ in Section 2 for $t_{0}=0$ and $t_{1}=T$. Note that the terminal states $y(0)$ and $y(T)$ are left free in the problem $(\bar{P}_{[0,T]})$. Recall that the solutions $(y(\cdot),u(\cdot))\in C([0,T];X)\times L^{2}(0,T;U)$ are considered in the mild sense, meaning that

[TABLE]

or equivalently,

[TABLE]

for each $\tau\in[0,T]$ and $\varphi\in D(A^{*})$, where $A^{*}:D(A^{*})\subset X^{*}\rightarrow X^{*}$ is the adjoint operator of $A$, and $\langle\cdot,\cdot\rangle_{X^{*},X}$ is the dual paring between $X$ and its dual space $X^{*}$.

Likewise, we say $(y(\cdot),u(\cdot))$ an admissible pair to the problem $(\bar{P}_{[0,T]})$ if it satisfies the state equation and the above state-control constraint. Assume that, for any $T>0$, $(\bar{P}_{[0,T]})$ has at least one optimal solution denoted by $(y^{T}(\cdot),u^{T}(\cdot))$, and we set

[TABLE]

Note that $\bar{J}^{T}$ does not depend on the optimal solution under consideration.

In the finite-dimensional case where $X=\mathbb{R}^{n}$ and $U=\mathbb{R}^{m}$, without loss of generality, we may take $A=0$, and then the control system is $\dot{y}(t)=f(y(t),u(t))$. We refer the reader to [12, 29] for the asymptotic behavior of optimal solutions of such optimal control problems with constraints on the terminal states.

Consider the static optimal control problem

[TABLE]

where the first equation means that

[TABLE]

As above, we assume that there exists at least one optimal solution $(y_{s},u_{s})$ of $(P_{s})$. Such existence results are as well standard, for instance in the case where $A$ is an elliptic differential operator (see [18, Chapter 3, Theorem 6.4]). We set

[TABLE]

Note that $\bar{J}_{s}$ does not depend on the optimal solution that is considered. Of course, uniqueness of the minimizer cannot be ensured in general because the problem is not assumed to be convex. Note that $(y_{s},u_{s})$ is admissible for the problem $(\bar{P}_{[0,T]})$ for any $T>0$, meaning that it satisfies the constraints and is a solution of the control system.

We next define the notion of dissipativity for the infinite-dimensional controlled system, which is originally due to [31] for finite-dimensional dynamics (see also related definitions in [12]). Recall that the continuous function $\alpha:[0,+\infty)\rightarrow[0,+\infty)$ with $\alpha(0)=0$ is said to be a $\mathcal{K}$-class function if it is monotone increasing.

Definition 1**.**

We say that $\{(\bar{P}_{[0,T]})\,\mid\,T>0\}$ is dissipative at an optimal stationary point $(y_{s},u_{s})$ with respect to the supply rate function

[TABLE]

if there exists a storage function $S:E\rightarrow\mathbb{R}$, locally bounded and bounded from below, such that, for any $T>0$, the dissipation inequality

[TABLE]

holds true, for any admissible pair $(y(\cdot),u(\cdot))$.

We say it is strictly dissipative at $(y_{s},u_{s})$ with respect to the supply rate function $\omega$ if there exists a $\mathcal{K}$-class function $\alpha(\cdot)$ such that, for any $T>0$, the strict dissipation inequality

[TABLE]

holds true, for any admissible pair $(y(\cdot),u(\cdot))$. The function $d(\cdot)=\alpha(\|(y(\cdot)-y_{s},u(\cdot)-u_{s})\|_{X\times U})$ in (3.3) is called the dissipation rate.

Although there are many possibly different notions of dissipativity introduced in the literature (such as the positivity or the local boundedness of the storage function in their definitions, cf., e.g., [5, Chapter 4]), they are proved to be equivalent in principle between with each other. Note that a storage function is defined up to an additive constant. We here define the storage function $S:E\rightarrow\mathbb{R}$ to take real values instead of positive real values. Since $S$ is assumed to be bounded from below, one could as well consider $S:E\rightarrow\mathbb{[}0,+\infty)$. We mention that no regularity is a priori required to define it. Actually, storage functions do possess some regularity properties, such as $C^{0}$ or $C^{1}$ regularity, under suitable assumptions. For example, the controllable and observable systems with positive transfer functions are dissipative with quadratic storage functions (see [5, Section 4.4.5] for instance).

When a system is dissipative with a given supply rate function, the question of finding a storage function has been extensively studied. This question is closely similar to the problem of finding a suitable Lyapunov function in the Lyapunov second method ensuring the stability of a system. For linear systems with a quadratic supply rate function, the existence of a storage function boils down to solve a Riccati inequality. In general, storage functions are closely related to viscosity solutions of a partial differential inequality, called a Hamilton-Jacobi inequality. We refer the reader to [5, Chapter 4] for more details on this subject.

An equivalent characterization of the dissipativity in [31] can be described by the so-called available storage, which is defined as

[TABLE]

where the $\sup$ is taken over all admissible pairs $(y(\cdot),u(\cdot))$ (meaning that satisfy the dynamic controlled system and state-control constraints) with initial value $y(0)=y$. In fact, for every $y\in E$, $S_{a}(y)$ can be seen as the maximum amount of “energy” which can be extracted from the system with initial state $y=y(0)$. It has been shown by Willems [31] that the problem $(\bar{P}_{[0,T]})$ is dissipative at $(y_{s},u_{s})$ with respect to the supply rate function $\omega(\cdot,\cdot)$ if and only if $S_{a}(y)$ is finite for every $y\in E$.

We provide a specific example of a (strictly) dissipative control system.

Example 2**.**

Let $\Omega\subset\mathbb{R}^{n}$ ($n\geq 1$) be a smooth and bounded domain, and let $\mathcal{D}\subset\Omega$ be a non-empty open subset. Denote by $\langle\cdot,\cdot\rangle$ and $\|\cdot\|$ the inner product and norm in $L^{2}(\Omega)$ respectively. For each $T>0$, consider the optimal control problem

[TABLE]

subject to

[TABLE]

Notice that the corresponding static problem has a unique solution $(0,0)$. We show that this problem is strictly dissipative at $(0,0)$ with respect to the supply rate

[TABLE]

In fact, integrating the heat equation by parts leads to

[TABLE]

for any $\tau\in[0,T]$. This, together with the definition of $\omega(\cdot,\cdot)$ above and the Poincaré inequality, indicates that the strict dissipation inequality

[TABLE]

holds with $\alpha(\gamma)=c\gamma^{2}$ for some constant $c>0$, and a storage function $S(\cdot)$ given by

[TABLE]

Thus, this problem has the strict dissipativity property at $(0,0)$.

3.2 Strict dissipativity implies measure-turnpike

Next, we introduce a rigorous definition of measure-turnpike for optimal control problems.

Definition 2**.**

We say that $\{(\bar{P}_{[0,T]})\,\mid\,T>0\}$ enjoys the measure-turnpike property at $(y_{s},u_{s})$ if, for every $\varepsilon>0$, there exists $\Lambda(\varepsilon)>0$ such that

[TABLE]

where

[TABLE]

We refer the reader to [7, 12, 33] (and references therein) for similar definitions. In this definition, the set $Q_{\varepsilon,T}$ measures the set of times at which the optimal trajectory and control stay outside an $\varepsilon$-neighborhood of $(y_{s},u_{s})$ for the strong topology. We stress that the measure-turnpike property defined above concerns both state and control. In the existing literature (see, e.g., [7]), the turnpike phenomenon is often studied only for the state, meaning that, for each $\varepsilon>0$, the (Lebesgue) measure of the set $\{t\in[0,T]\,\mid\,\|y^{T}(t)-y_{s}\|_{X}>\varepsilon\}$ is uniformly bounded for any $T>0$.

In the following result, we establish the measure-turnpike property for optimal solutions of $(\bar{P}_{[0,T]})$ (as well for $(\bar{P}_{[t_{0},t_{1}]})$) under the strict dissipativity assumption, as the parameter $T$ goes to infinity. This implies that any optimal solution $(y^{T}(\cdot),u^{T}(\cdot))$ of $(\bar{P}_{[0,T]})$ remains essentially close to some optimal solution $(y_{s},u_{s})$ of $(P_{s})$. Our results can be seen in the stream of the recent works [12, 22, 23, 29].

Theorem 2**.**

Let $E$ be a bounded subset of $X$.

1. (i).

If $\{(\bar{P}_{[0,T]})\,\mid\,T>0\}$ is dissipative at $(y_{s},u_{s})$ with respect to the supply rate function $\omega(\cdot,\cdot)$ given by (3.1), then

[TABLE]

as $T\rightarrow+\infty$. 2. (ii).

If $\{(\bar{P}_{[0,T]})\,\mid\,T>0\}$ is strictly dissipative at $(y_{s},u_{s})$ with respect to the supply rate function $\omega(\cdot,\cdot)$ given by (3.1), then it satisfies the measure-turnpike property at $(y_{s},u_{s})$.

Remark 8**.**

Note that $(\bar{P}_{[0,T]})$ is defined without any constraints on the terminal states. However, under appropriate controllability assumptions (similar to $(H_{3})$ in Section 2), one can also treat the case of terminal state constraint $R(\cdot)=0$ (see also [12] and [13]).*

Remark 9**.**

From Theorem 2, we see that strict dissipativity is sufficient for the measure-turnpike property for the optimal control problem. This fact was observed in the previous works [9, 12, 13]. For the converse statements, i.e., results which show that the turnpike property implies strict dissipativity, we refer the reader to [14] and [12]. In [14], the authors first defined a turnpike-like behavior concerning all trajectories whose associate cost is close to the optimal one. This behavior is stronger than the measure-turnpike property, which only concerns the optimal trajectories. Then, the implication “ turnpike-like behavior $\Rightarrow$ strict dissipativity” was proved in [14]. Besides, the implication “ exact turnpike property $\Rightarrow$ strict dissipativity along optimal trajectories” was shown in [12], where the exact turnpike property means that the optimal solutions have to remain exactly at an optimal steady-state for most part of the long-time horizon. *

Proof of Theorem 2..

We first prove the second point of the theorem. Let $T>0$ and let $(y^{T}(\cdot),u^{T}(\cdot))$ be any optimal solution of the problem $(\bar{P}_{[0,T]})$. By the strict dissipation inequality (3.3) applied to $(y^{T}(\cdot),u^{T}(\cdot))$, we have

[TABLE]

Note that \alpha\big{(}\|(y^{T}(t)-y_{s},u^{T}(t)-u_{s})\|_{X\times U}\big{)}\geq\alpha(\varepsilon) whenever $t\in Q_{\varepsilon,T}$, where $Q_{\varepsilon,T}$ is defined by (3.5). Since $E\subset X$ is a bounded subset and $S(\cdot)$ is locally bounded, there exists $M>0$ such that $|S(y)|\leq M$ for every $y\in E$. Therefore, it follows from (3.7) that

[TABLE]

On the other hand, noting that $(y_{s},u_{s})$ is admissible for $(\bar{P}_{[0,T]})$ for any $T>0$, we have

[TABLE]

This, combined with (3.8), leads to $|Q_{\varepsilon,T}|\leq\frac{2M}{\alpha(\varepsilon)}$ for every $T>0$. The second point of the theorem follows.

Let us now prove the first point. On the one hand, it follows from (3.9) that

[TABLE]

By the dissipation inequality (3.2) applied to any optimal solution $(y^{T}(\cdot),u^{T}(\cdot))$ of $(\bar{P}_{[0,T]})$, we get

[TABLE]

which leads to

[TABLE]

Since $E$ is a bounded subset in $X$ and since the storage function $S(\cdot)$ is locally bounded and bounded below, we infer that

[TABLE]

Then (3.6) follows. ∎

Remark 10**.**

The above proof borrows ideas from [13, Theorem 5.3] and [12]. We used in a crucial way the fact that any solution of the steady-state problem $(P_{s})$ is admissible for the problem $(\bar{P}_{[0,T]})$ under consideration. This is due to the fact that the terminal states are let free in $(\bar{P}_{[0,T]})$. Note that we only use the boundedness of $y^{T}(0)$ in the proof.*

3.3 Dissipativity and Assumption (H)

Under the (strict) dissipativity property, we can verify the abstract Assumption (H) for the autonomous case in Section 2.

Proposition 1**.**

Assume that, for any $t_{0}$ and $t_{1}$, the problem $(\bar{P}_{[t_{0},t_{1}]})$ is dissipative at $(y_{s},u_{s})$ with the supply rate $\omega(y,u)=f^{0}(y,u)-f^{0}(y_{s},u_{s})$, and the associated storage function $S(\cdot)$ is bounded on $E$. Then

• (i).

$\bar{J}_{[t_{0},+\infty)}=\bar{J}_{s}$, $\forall t_{0}\in\mathbb{R}$.

• (ii).

There exists a turnpike set $\mathcal{T}=\{y_{s}\}$ such that $(H_{1})$ and $(H_{2})$ are satisfied.

• (iii).

Moreover, if $(H_{3})$ is satisfied and it is strictly dissipative at $(y_{s},u_{s})$ with dissipation rate $d(\cdot)=\alpha(\|y(\cdot)-y_{s}\|_{X})$, then $(H_{4})$ is satisfied with $\beta(\cdot)=\alpha(\cdot)$ and $\text{dist}\,(y,\mathcal{T})=\|y-y_{s}\|_{X}$.

Proof.

$(i)$. With a slight modification, the proof is the same as that of the first point of Theorem 2.

$(ii)$. Since $(y_{s},u_{s})$ is an equilibrium point, the constant pair $(y_{s},u_{s})$ is admissible on any time interval. By the definition,

[TABLE]

This, along with $(i)$, indicates that

[TABLE]

Hence, the assumptions $(H_{1})$ and $(H_{2})$ hold.

$(iii).$ Let $(\tilde{y}(\cdot),\tilde{u}(\cdot))$ be an optimal solution to the problem $(P_{[t_{0},t_{1}]})$. Then, by the strict dissipativity property we have

[TABLE]

Which is equivalent to

[TABLE]

Because

[TABLE]

The last inequality, along with (3.10), indicates that

[TABLE]

As the storage function $S(\cdot)$ is bounded on $E$, by $(i)$ and (2.2) we infer

[TABLE]

Hence, the sum of last three terms in (3.11) is an infinitesimal quantity as $t_{1}\rightarrow+\infty$, and thus $(H_{4})$ holds.

∎

Remark 11**.**

Proposition 1 explains the role of dissipativity in the general turnpike phenomenon. It reflects that dissipativity allows one to identify the limit value $\bar{J}_{[t_{0},+\infty)}$, that dissipativity implies $(H_{1})$ and $(H_{2})$, and that strict dissipativity, plus $(H_{3})$, implies $(H_{4})$. Recall that $(H_{3})$ is a controllability assumption.*

3.4 Some comments on the periodic turnpike phenomenon

Inspired from [31] and [35], we introduce the concept of (strict) dissipativity with respect to a periodic trajectory. Let $A(\cdot)$, $f(\cdot)$ and $f^{0}(\cdot)$ be periodic in time with a period $\Pi>0$.

Definition 3**.**

We say the problem $(\bar{P}_{[t_{0},t_{1}]})$ is dissipative with respect to a $\Pi$-periodic trajectory $(\hat{y}(\cdot),\hat{u}(\cdot))$ with respect to the supply rate function

[TABLE]

if there exists a locally bounded and bounded from below storage function $S:\mathbb{R}\times E\rightarrow\mathbb{R}$, $\Pi$-periodic in time, such that

[TABLE]

for any admissible pair $(y(\cdot),u(\cdot))$. If, in addition, there exists a $\mathcal{K}$-class function $\alpha(\cdot)$ such that

[TABLE]

we say it is strictly dissipative with respect to a $\Pi$-periodic trajectory $(\hat{y}(\cdot),\hat{u}(\cdot))$.

The notion of (strict) dissipativity with respect to a periodic trajectory in Definition 3 allows one to identify the optimal control problem $(\bar{P}_{[t_{0},t_{1}]})$ as a periodic one. Consider the periodic optimal control problem

[TABLE]

We assume that $(\bar{P}_{\textrm{per}})$ has at least one periodic optimal solution $(\bar{y}(\cdot),\bar{u}(\cdot))$ on $[0,\Pi]$ (see e.g. [3] for the existence of periodic optimal solutions), and we set $\bar{J}_{\textrm{per}}=\frac{1}{\Pi}\int_{0}^{\Pi}f^{0}(t,\bar{y}(t),\bar{u}(t))\,dt$ (optimal value of $(\bar{P}_{\textrm{per}})$). Let us extend $(\bar{y}(\cdot),\bar{u}(\cdot))$ in $\mathbb{R}$ by periodicity. Likewise, we have the following result.

Proposition 2**.**

Assume that, for any $t_{0}$ and $t_{1}$, the problem $(\bar{P}_{[t_{0},t_{1}]})$ is dissipative with respect to the $\Pi$-periodic optimal trajectory $(\bar{y}(\cdot),\bar{u}(\cdot))$, with the supply rate $\omega(t,y,u)=f^{0}(t,y,u)-f^{0}(t,\bar{y}(t),\bar{u}(t))$, and the associated storage function $S(\cdot)$ is bounded on $E$ for all times. Then

• (i).

$\bar{J}_{[t_{0},+\infty)}=\bar{J}_{per}$, $\forall t_{0}\in\mathbb{R}$.

• (ii).

There exists a turnpike set $\mathcal{T}=\{\bar{y}(t)\ \mid\ t\in[0,\Pi]\}$ such that $(H_{1})$ and $(H_{2})$ are satisfied.

• (iii).

Moreover, if $(H_{3})$ is satisfied and $(\bar{P}_{[t_{0},t_{1}]})$ is strictly dissipative with respect to the $\Pi$-periodic trajectory $(\bar{y}(\cdot),\bar{u}(\cdot))$, with dissipation rate $\alpha(\cdot)$, then $(H_{4})$ is satisfied with $\beta(\cdot)=\alpha(\cdot)$ and $\text{dist}\,(y,\mathcal{T})=\min_{t\in[0,\Pi]}\|y-\bar{y}(t)\|_{X}$.

Proof.

We only show the proof of $(i)$, as the rest is similar to the arguments in the proof of Proposition 1.

Since $(\bar{y}(\cdot),\bar{u}(\cdot))$ is an admissible trajectory in $[t_{0},t_{0}+k\Pi]$ for any $k\in\mathbb{N}$, we have $\bar{J}_{[t_{0},+\infty)}\leq\bar{J}_{\textrm{per}}$. Let us prove the converse inequality. By the periodic dissipativity in Definition 3, we have

[TABLE]

for any admissible trajectory $(y(\cdot),u(\cdot))$. Since $\bar{J}_{\textrm{per}}=\frac{1}{\Pi}\int_{0}^{\Pi}f^{0}(t,\bar{y}(t),\bar{u}(t))\,dt$, it follows that

[TABLE]

Letting $k$ tend to infinity, and taking the infimum over all possible admissible trajectories, we get that $\bar{J}_{\textrm{per}}\leq\bar{J}_{[t_{0},+\infty)}$.

∎

4 Relationship with (strict) strong duality

After having detailed a motivating example in Section 4.1, we recall in Section 4.2 the notion of (strict) strong duality, and we establish in Section 4.3 that strict strong duality implies strict dissipativity (and thus measure-turnpike according to Section 3.2).

4.1 A motivating example

To illustrate the effect of Lagrangian function associated with the static problem when one derives the measure-turnpike property for the evolution control system, we consider the simplest model of heat equation with control constraints.

Let $\Omega\subset\mathbb{R}^{n}$, $n\geq 1$, be a bounded domain with a smooth boundary $\partial\Omega$, and let $\mathcal{D}\subset\Omega$ be a non-empty open subset. Throughout this subsection, we denote by $\langle\cdot,\cdot\rangle$ and $\|\cdot\|$ the inner product and norm in $L^{2}(\Omega)$, respectively; by $\chi_{\mathcal{D}}$ the characteristic function of $\mathcal{D}$. For any $T>0$, consider the optimal control problem for the heat equation with pointwise control constraints:

[TABLE]

subject to

[TABLE]

where $y_{d}\in L^{2}(\Omega)$, $y_{0}\in L^{2}(\Omega)$ and

[TABLE]

with $u_{a}$ and $u_{b}$ being in $L^{2}(\Omega)$. Assume that $(y^{T}(\cdot),u^{T}(\cdot))$ (the optimal pair obviously depends on the time horizon) is the unique optimal solution. We want to study the long time behavior of optimal solutions, i.e., the optimal pair stays in a neighborhood of a static optimal solution at most of the time horizon.

As before, we consider the static optimal control problem stated below

[TABLE]

subject to

[TABLE]

Assume that $(y_{s},u_{s})$ is the unique optimal solution to $(P_{s})$.

For this purpose, given every $\varepsilon>0$, we define the set

[TABLE]

which measures the time at which the optimal pair is outside of the $\varepsilon$-neighborhood of $(y_{s},u_{s})$.

Proposition 3**.**

The following convergence hold

[TABLE]

Moreover, for each $\varepsilon>0$, it holds true that

[TABLE]

i.e., the measure-turnpike property holds.

Proof.

The key point of the proof is to show that

[TABLE]

where $C$ is a constant independent of $T$. Once the inequality (4.1) is proved, the desired results hold automatically. To prove (4.1), the remaining part of the proof is proceeded into several steps as follows.

Step 1. We first introduce a Lagrangian function for the above stationary problem. According to the Karusch-Kuhn-Tucker (KKT for short) optimality conditions (see, e.g., [27, Theorem 2.29]), there are functions $p_{s}$, $\mu_{a}$ and $\mu_{b}$ in $L^{2}(\Omega)$ such that

[TABLE]

Now, we define the associated Lagrangian function $L:H_{0}^{1}(\Omega)\times L^{2}(\Omega)\rightarrow\mathbb{R}$ by setting

[TABLE]

From the above-mentioned KKT optimality conditions, we can see that

[TABLE]

[TABLE]

[TABLE]

Since $L$ is a quadratic form, the Taylor expansion is

[TABLE]

which means that

[TABLE]

Step 2. Noting that $\mu_{a}\geq 0$ and $\mu_{b}\geq 0$, we obtain from (4.2) and (4.3) that for each $(y,u)\in H_{0}^{1}(\Omega)\times\mathcal{U}_{ad}$,

[TABLE]

Since $(y^{T}(t),u^{T}(t))\in H^{1}_{0}(\Omega)\times\mathcal{U}_{ad}$ for a.e. $t\in(0,T)$, we get from (4.4) that

[TABLE]

Integrating the above inequality over $(0,T)$ and then multiplying the resulting by $1/T$, we have

[TABLE]

Observe that

[TABLE]

This, along with (4.5), implies that

[TABLE]

By the standard energy estimate for non-homogeneous heat equations, there is a constant $C>0$ (independent of $T>0$) such that

[TABLE]

Hence, by the Cauchy-Schwarz inequality we have

[TABLE]

This, together with (4.6), indicates that

[TABLE]

Step 3. We claim that

[TABLE]

Indeed, since $u_{s}$ is always an admissible control for the problem $(P^{T})$, it holds that

[TABLE]

where $y(\cdot;u_{s})$ is the solution to

[TABLE]

It can be readily checked that

[TABLE]

Which in turn, together with (4.10), implies that (4.9).

Step 4. End of the proof for the inequality $\eqref{i7}$. We obtain immediately from (4.8) and (4.9) that

[TABLE]

as well as

[TABLE]

which is equivalent to the inequality (4.1). ∎

Remark 12**.**

Notice that the inequality (4.1) is stronger than the weak turnpike property (2.3) for $\mathcal{T}=\{y_{s}\}$. The proof above yields the convergence result for the long-time horizon control problems towards to the steady-state one in the measure-theoretical sense. It is an improved version of the case of time-independent controls [23, Section 4].*

Remark 13**.**

We remark that, in the steps 2 and 3 of the proof of Proposition 3, we have used the exponential stabilization of the heat equation to derive the upper bounds (4.7) and (4.11). See also Remark 6.*

4.2 What is (strict) strong duality

In the above proof of Proposition 3, we have seen an important role played by the Lagrangian (4.3), which is closely related to the notion of strict strong duality introduced below. We recall that the notion of strong duality, well known in optimization (see, e.g., [4]).

Definition 4**.**

We say that the static problem $(P_{s})$ (in Section 3.1) has the strong duality property if there exists $\varphi_{s}\in D(A^{*})$ (Lagrangian multiplier) such that $(y_{s},u_{s})$ minimizes the Lagrangian function $L(\cdot,\cdot,\varphi_{s}):E\times F\rightarrow\mathbb{R}$ defined by

[TABLE]

We say $(P_{s})$ has the strict strong duality property if there exists a $\mathcal{K}$-class function $\alpha(\cdot)$ such that

[TABLE]

for all $(y,u)\in E\times F$.

Remark 14**.**

Note that $L(y_{s},u_{s},\varphi_{s})=\bar{J}_{s}$. If $(y_{s},u_{s})$ is the unique minimizer of the Lagrangian function $L(\cdot,\cdot,\varphi_{s})$, and if $E\times F$ is compact in $X\times U$, then $(P_{s})$ enjoys the strict strong duality property. However, it is generally a very strong assumption that $L(\cdot,\cdot,\varphi_{s})$ has a unique minimizer. Note that uniqueness of minimizers for elliptic optimal control problems is still a long outstanding and difficult problem (cf., e.g., [27]).*

In finite dimension, strong duality is introduced and investigated in optimization problems for which the primal and dual problems are equivalent. The notion of strong duality is closely related to the saddle point property of the Lagrangian function associated with the primal optimization problem (see, e.g., [4, 27]). Note that Slater’s constraint qualification (also known as “interior point” condition) is a sufficient condition ensuring strong duality for a convex problem, and note that, when the primal problem is convex, the well known Karusch-Kuhn-Tucker conditions are also sufficient conditions ensuring strong duality (see [4, Chapter 5]). Similar assumptions are also considered for other purposes in the literature (see, for example, [6, Assumption 1], [7, Assumption 4.2 (ii)] and [8, Assumption 2].)

In infinite dimension, however, the usual strong duality theory (for example, the above-mentioned Slater condition) cannot be applied because the underlying constraint set may have an empty interior. The corresponding strong dual theory, as well as the existence of Lagrange multipliers associated to optimization problems or to variational inequalities, have been developed only quite recently in [10]. The strict strong duality property is closely related to the second-order sufficient optimality condition, which guarantees the local optimality of $(y_{s},u_{s})$ for the problem $(P_{s})$ (see, e.g., [27]).

We provide hereafter two examples satisfying the strict strong duality property.

Example 3**.**

Consider the static optimal control problem

[TABLE]

with $A\in\mathbb{R}^{n\times n},B\in\mathbb{R}^{n\times m}$, $f^{0}(\cdot,\cdot)$ a strictly convex function, $E$ and $F$ convex, bounded and closed subsets of $\mathbb{R}^{n}$ and of $\mathbb{R}^{m}$, respectively.

Assume that Slater’s condition holds, i.e., there exists an interior point $(\tilde{y},\tilde{u})$ of $E\times F$ such that $A\tilde{y}+B\tilde{u}=0$. Recall that the Lagrangian function $L:E\times F\times\mathbb{R}^{n}\rightarrow\mathbb{R}$ is given by $L(y,u,\varphi)=f^{0}(y,u)+\langle\varphi,Ay+Bu\rangle_{\mathbb{R}^{n}}$. Let $(y_{s},u_{s})$ be the unique optimal solution. It follows from the Slater condition that there exists a Lagrangian multiplier $\varphi_{s}\in\mathbb{R}^{n}$ such that (see, e.g., [4, Section 5.2.3])

[TABLE]

The strict inequality is due to the strict convexity of the cost function $f^{0}$. Setting

[TABLE]

we have $\widetilde{L}(y_{s},u_{s})=0$ and $\widetilde{L}(y,u)>0$ for all $(y,u)\in E\times F\setminus\{(y_{s},u_{s})\}$.

We claim that

[TABLE]

for some $\mathcal{K}$-class function $\alpha(\cdot)$. Indeed, since $E\times F$ is compact in $\mathbb{R}^{n+m}$, without loss of generality, we assume that $E\times F\subset B_{r}(y_{s},u_{s})$ with $r>0$, where

[TABLE]

Since the function $\widetilde{L}(\cdot,\cdot)$ is continuous, we define

[TABLE]

and $\alpha(\gamma)\equiv\alpha(r)$ when $\gamma>r$. It is easy to check that the inequality (4.12) holds with the $\mathcal{K}$-class function $\alpha(\cdot)$ given above. This means that the static problem has the strict strong duality property.

Example 4**.**

Let $\Omega\subset\mathbb{R}^{3}$ be a bounded domain with a smooth boundary $\partial\Omega$. Given any $y_{d}\in L^{2}(\Omega)$, we consider the static optimal control problem

[TABLE]

over all $(y,u)\in H^{1}_{0}(\Omega)\times L^{2}(\Omega)$ satisfying

[TABLE]

Let $(y_{s},u_{s})$ be an optimal solution of this problem. According to first-order necessary optimality conditions (see, e.g., [17, Chapter 1] or [27, Chapter 6, Section 6.1.3]), there exists an adjoint state $\varphi_{s}\in H^{2}(\Omega)\cap H^{1}_{0}(\Omega)$ satisfying

[TABLE]

such that $u_{s}=\varphi_{s}$. Moreover, since $\varphi_{s}$ is a Lagrangian multiplier associated with $(y_{s},u_{s})$ for the Lagrangian function $L(\cdot,\cdot,\varphi_{s}):H^{1}_{0}(\Omega)\times L^{2}(\Omega)\rightarrow\mathbb{R}$ defined by

[TABLE]

we have

[TABLE]

It means that $(P_{s})$ has the strong duality property.

Next, we claim that it holds the strict strong duality property under the condition that $\|y_{d}\|_{L^{2}(\Omega)}$ is small enough. Notice that

[TABLE]

Now, assuming that the norm of the target $y_{d}$ is small enough guarantees the smallness of $(y_{s},u_{s})$, which consequently belongs to a ball $B_{r}$ in $H_{0}^{1}(\Omega)\times L^{2}(\Omega)$, centered at the origin and with a small radius $r>0$. Moreover, by elliptic regularity, we deduce that the norms of $y_{s}$ and $\varphi_{s}$ are small in $H^{2}(\Omega)\cap L^{\infty}(\Omega)$ (see [23, Section 3]). For the Lagrangian function $L(\cdot,\cdot,\varphi_{s})$ defined above, its first-order Fréchet derivative is

[TABLE]

and its second-order Fréchet derivative is

[TABLE]

whenever $(y,u)\in B_{r}$ (see, for instance, [27, Chapter 6, pp. 337-338]). Note that

[TABLE]

for all $(y,u)\in B_{r}$. This, together with (4.13), (4.14) and the smallness of $(y_{s},\varphi_{s})$ in $L^{\infty}(\Omega)$, implies that

[TABLE]

which proves the above claim.

Remark 15**.**

Similar to second order gap conditions for local optimality [27], the positive semi-definiteness of Hessian matrix of the Hamiltonian is a necessary condition for the local optimality, while its positive definiteness is a sufficient condition for the local optimality. The latter is also known as the strengthened Legendre-Clebsch condition.*

4.3 Strict strong duality implies strict dissipativity

In this subsection, by means of strict strong duality, we extend Proposition 3 to general optimal control problems. More precisely, we establish sufficient conditions, in terms of (strict) strong duality for $(P_{s})$, under which (strict) dissipativity holds true with a specific storage function for $(\bar{P}_{[0,T]})$ in Section 3.1. As seen in Theorem 2, strict dissipativity implies measure-turnpike.

Theorem 3**.**

Let $E$ be a bounded subset of $X$. Then, strong duality (resp., strict strong duality) for $(P_{s})$ implies dissipativity (resp., strict dissipativity) for $(\bar{P}_{[0,T]})$, with the storage function given by $S(y)=-\langle\varphi_{s},y\rangle_{X^{*},X}$ for every $y\in E$. Consequently, $(\bar{P}_{[0,T]})$ has the measure-turnpike property under the strict strong duality property.

Proof.

It suffices to prove that strong duality for $(P_{s})$ implies dissipativity for $(\bar{P}_{[0,T]})$ (the proof with the “strict” additional property is similar with only minor modifications).

By the definition of strong duality, there exists a Lagrangian multiplier $\varphi_{s}\in D(A^{*})$ such that $L(y_{s},u_{s},\varphi_{s})\leq L(y,u,\varphi_{s})$ for all $(y,u)\in E\times F$, which means that

[TABLE]

Let $T>0$. Assume that $(y(\cdot),u(\cdot))$ is an admissible pair for the problem $(\bar{P}_{[0,T]})$. Then,

[TABLE]

Integrating the above inequality over $(0,\tau)$, with $0<\tau\leq T$, leads to

[TABLE]

Notice that $(y(\cdot),u(\cdot))$ satisfies the state equation in the problem $(\bar{P}_{[0,T]})$, we have

[TABLE]

This, together with (4.15), leads to

[TABLE]

Set $S(y)=-\langle\varphi_{s},y\rangle_{X^{*},X}$ for every $y\in E$. Since $E$ is a bounded subset of $X$, we see that $S(\cdot)$ is locally bounded and bounded from below. Therefore, we infer that $\{(\bar{P}_{[0,T]})\,\mid\,T>0\}$ has the dissipativity property. ∎

Remark 16**.**

Strong duality and dissipativity are equivalent in some situations:

• •

*On one hand, we proved above that strong duality (resp. strict strong duality) implies dissipativity (resp., strict dissipativity). We refer also the reader to [12, Lemma 3] for a closely related result. *

• •

On the other hand, it is easy to see that, if the storage function $S(\cdot)$ is continuously Fréchet differentiable, then strong duality (resp., strict strong duality) is the infinitesimal version of the dissipative inequality (3.2) (resp., of (3.3)). For this point, we also mention that [13, Assumption 5.2] is a discrete version of strict dissipativity, and that [12, Inequality (14)] is the infinitesimal version of strict dissipativity for the continuous system when the storage function is differentiable.

5 Conclusions and further comments

In this paper, we first have proved a general turnpike phenomenon around a set holds for optimal control problems with terminal state constraints in an abstract framework. Next, we have obtained the following auxiliary result:

strict strong duality $\Rightarrow$ strict dissipativity $\Rightarrow$ measure-turnpike property.

We have also used dissipativity to identify the long-time limit of optimal values.

Now, several comments and perspectives are in order.

Measure-turnpike versus exponential turnpike.

In the paper [28], we establish the exponential turnpike property for general classes of optimal control problems in infinite dimension that are similar to the problem $(\bar{P}_{[0,T]})$ investigated in the present paper, but with the following differences:

• (i)

$E=X$ and $F=U$;

• (ii)

$y(0)=y_{0}\in X$.

The item (i) means that, in [28], we consider optimal control problems without any state or control constraint. Under the additional assumption made in (ii), we are then able to apply the Pontryagin maximum principle in Banach spaces (see [18]), thus obtaining an extremal system that is smooth, which means in particular that the extremal control is a smooth function of the state and of the adjoint state. This smooth regularity is crucial in the analysis done in [28] (see also [29]), consisting of linearizing the extremal system around an equilibrium point, which is itself the optimal solution of an associated static optimal control problem, and then of analyzing from a spectral point of view the hyperbolicity properties of the resulting linear system. Adequately interpreted, this implies the local exponential turnpike property, saying that

[TABLE]

for every $t\in[0,T]$, for some constants $\mu,c>0$ not depending on $T$, where $\lambda^{T}$ is the adjoint state coming from the Pontryagin maximum principle. There are many examples of control systems for which the measure-turnpike holds but not exponential turnpike (see, for instance, Example  3). The exponential turnpike property is much stronger than the measure-turnpike property, not only because it gives an exponential estimate on the control and the state, instead of the softer estimate in terms of Lebesgue measure, but also because it gives the closeness property for the adjoint state. This leads us to the next comment.

Turnpike on the adjoint state.

As mentioned above, the exponential turnpike property established in [28] holds as well for the adjoint state coming from the application of the Pontryagin maximum principle. This property is particularly important when one wants to implement a numerical shooting method in order to compute the optimal trajectories. Indeed, the exponential closeness property of the adjoint state to the optimal static adjoint allows one to successfully initialize a shooting method, as chiefly explained in [29] where an appropriate modification and adaptation of the usual shooting method has been described and implemented.

The flaw of the linearization approach developed in [28] is that it does not a priori allow to take easily into account some possible control constraints (without speaking of state constraints).

The softer approach developed in the present paper leads to the weaker property of measure-turnpike, but permits to take into account some state and control constraints.

However, under the assumption (ii) above, one can as well apply the Pontryagin maximum principle, and thus obtain an adjoint state $\lambda^{T}$. Due to state and control constraints, of course, one cannot expect that the extremal control $u^{T}$ be a smooth function of $y^{T}$ and $\lambda^{T}$, but anyway our approach by dissipativity is soft enough to yield the measure-turnpike property for the optimal state $y^{T}$ and for the optimal control $u^{T}$. Now, it is an open question to know whether the measure-turnpike property holds or not for the adjoint state $\lambda^{T}$. As mentioned above, having such a result is particularly important in view of numerical issues.

Local versus global properties.

It is interesting to stress on the fact that Theorem 2 (saying that strict dissipativity implies measure-turnpike) is of global nature, whereas Theorem 3 (saying that strict strong duality implies strict dissipativity) is rather of local nature. This is because, as soon as Lagrangian multipliers enter the game, except under strong convexity assumptions this underlies that one is performing reasonings that are local, such as applying first-order conditions for optimality. Therefore, although Theorem 3 provides a sufficient condition ensuring strict dissipativity and thus allowing one to apply the result of Theorem 2, in practice showing strict strong duality can in general only be done locally. In contrast, dissipativity is a much more general property, which is global in the sense that it reflects a global qualitative behavior of the dynamics, as in the Lyapunov theory. We insist on this global picture because this is also a big difference with the results of [28, 29] on exponential turnpike, that are purely local and require smallness conditions. Here, in the framework of Theorem 2, no smallness condition is required. The price to pay however is that one has to know a storage function, ensuring strict dissipativity. In practical situations this is often the case and storage functions often represent an energy that has a physical meaning.

Semilinear heat equation.

We end the paper with a still open problem, related to the above-mentioned smallness condition. Continuing with Example 4, given any $y_{d}\in L^{2}(\Omega)$ we consider the evolution optimal control problem

[TABLE]

over all possible solutions of

[TABLE]

such that $(y(t),u(t))\in H_{0}^{1}(\Omega)\times L^{2}(\Omega)$ for almost every $t\in(0,T)$. It follows from Example 4 and Theorem 3 that the problem is dissipative at an optimal stationary point $(y_{s},u_{s})$ with the storage function $S(y)=-\langle\varphi_{s},y\rangle_{L^{2}(\Omega),L^{2}(\Omega)}$. Under the additional smallness condition on $\|y_{d}\|_{L^{2}(\Omega)}$, the strict strong duality holds and thus the measure-turnpike property follows. As said above, this assumption reflects the fact that Theorem 3 is rather of a local nature. However, due to the fact that the nonlinear term in (5.1) has the “right sign”, we do not know how to take advantage of this monotonicity of the control system (5.1) to infer the measure-turnpike property. It is interesting to compare this result with [23, Theorem 3.1], where the authors used a subtle analysis of optimality systems to establish an exponential turnpike property, under the same smallness condition. The question of whether the turnpike property actually holds or not for optimal solutions but without the smallness condition on the target, is still an interesting open problem.

Acknowledgment. We would like to thank Prof. Enrique Zuazua for fruitful discussions and valuable suggestions on this subject. We acknowledge the financial support by the grant FA9550-14-1-0214 of the EOARD-AFOSR. The second author was partially supported by the National Natural Science Foundation of China under grants 11501424 and 11371285.