Optimal control of linear systems with limited control actions: threshold-based event-triggered control

TL;DR

This paper addresses the challenge of limited control message transmission in linear-quadratic optimal control by proposing threshold-based event-triggered mechanisms, exact solutions, heuristics, and stability analysis for receding horizon control.

Contribution

It introduces an exact solution to a non-convex control problem and develops efficient heuristics, along with stability analysis for event-triggered receding horizon control.

Findings

Exact solution via quadratic programs

Heuristic algorithms reduce computational complexity

Stability of receding horizon control established

Abstract

We consider a finite-horizon linear-quadratic optimal control problem where only a limited number of control messages are allowed for sending from the controller to the actuator. To restrict the number of control actions computed and transmitted by the controller, we employ a threshold-based event-triggering mechanism that decides whether or not a control message needs to be calculated and delivered. Due to the nature of threshold-based event-triggering algorithms, finding the optimal control sequence requires minimizing a quadratic cost function over a non-convex domain. In this paper, we firstly provide an exact solution to the non-convex problem mentioned above by solving an exponential number of quadratic programs. To reduce computational complexity, we, then, propose two efficient heuristic algorithms based on greedy search and the Alternating Direction Method of Multipliers (ADMM) method. Later, we consider a receding horizon control strategy for linear systems controlled by event-triggered controllers, and we also provide a complete stability analysis of receding horizon control that uses finite horizon optimization in the proposed class. Numerical examples testify to the viability of the presented design technique.

Equations

$$\mathcal{G}:\mathbf{x}(t+1)=A\mathbf{x}(t)+B\mathbf{u}(t),\mathbf{x}(0)={\mathbf{x}}_0,$$

$${\mathcal{C}}_0\triangleq \{\mathbf{x}\in {\mathbb{R}}^n:\parallel \mathbf{x}{\parallel }_{\infty }<\epsilon \},$$

$$\mathbf{u}(t)=0_m,\forall t\in \mathcal{T},$$

$$\mathcal{T}\triangleq \{t\in {\mathbb{N}}_0;t<N:\mathbf{x}(t)\in {\mathcal{C}}_0\}.$$

$$J(\bm{x}(0),\bm{\pi})=\bm{x}^{\intercal}(N)P\bm{x}(N)\\ +\sum_{t=0}^{N-1}\big{(}\bm{x}^{\intercal}(t)Q\bm{x}(t)+\bm{u}^{\intercal}(t)R\bm{u}(t)\big{)}\;,$$

$$\displaystyle\begin{array}[]{rll}J^{\star}(\bm{x}(0)):&\stackrel{{\scriptstyle[}}{{\bm{}}}{\pi}]{}{\text{minimize}}&J(\bm{x}(0),\bm{\pi})\\ &\text{subject to}&\bm{x}(t+1)=A\bm{x}(t)+B\bm{u}(t),\\ &&\hskip 31.29802pt\forall t\in\{0,\cdots,N-1\},\\ &&\bm{x}(0)=\bm{x}_{0},\\ &&\bm{u}(t)=\mathbf{0}_{m},\quad\forall t\in\mathcal{T}.\end{array}$$

$$\displaystyle\begin{array}[]{rll}J_{\mathcal{T}}^{\star}(\bm{x}(0)):&\stackrel{{\scriptstyle[}}{{\bm{}}}{\pi}]{}{\text{minimize}}&J(\bm{x}(0),\bm{\pi})\\ &\text{subject to}&\bm{x}(t+1)=A\bm{x}(t)+B\bm{u}(t),\\ &&\hskip 31.29802pt\forall t\in\{0,1,\cdots,N-1\},\\ &&\bm{x}(t)\in\mathcal{C}_{0},~{}\forall t\in\mathcal{T},\\ &&\bm{x}(t)\in\mathbb{R}^{n}\setminus\mathcal{C}_{0},~{}\forall t\notin\mathcal{T},\\ &&\bm{u}(t)=\bm{0}_{m},~{}\forall t\in\mathcal{T},\\ &&\bm{x}(0)=\bm{x}_{0},\end{array}$$

$$\displaystyle\begin{array}[]{rll}J^{\star}(\bm{x}(0)):&\stackrel{{\scriptstyle[}}{{\mathcal{}}}{T}\in\mathsf{P}(\mathcal{N})]{}{\text{minimize}}&J_{\mathcal{T}}^{\star}(\bm{x}(0))\;.\end{array}$$

$$\mathcal{S}\triangleq \bigcup _{p=1}^{2n}{\mathcal{C}}_p,$$

$$\displaystyle\mathcal{C}_{p}\triangleq\big{\{}\bm{x}\in\mathbb{R}^{n}:T_{p}\bm{x}\leq\bm{d}\big{\}}\;,$$

$$\sigma (t)=\{\begin{array}{ll}0& \text{if}t\in \mathcal{T},\\ p& \text{otherwise},\end{array}$$

$$\displaystyle\begin{array}[]{rll}J_{\Sigma}^{\star}(\bm{x}(0)):&\stackrel{{\scriptstyle[}}{{\bm{}}}{\pi}]{}{\text{minimize}}&J(\bm{x}(0),\bm{\pi})\\ &\text{subject to}&\bm{x}(t+1)=A\bm{x}(t)+B\bm{u}(t),\\ &&\hskip 17.07164pt\forall t\in\{0,1,\cdots,N-1\},\\ &&\bm{x}(t)\in\mathcal{C}_{\sigma(t)},~{}\forall\sigma(t)\in\Sigma,\\ &&\bm{u}(t)=\bm{0}_{m},~{}\forall t\in\mathcal{T},\\ &&\bm{x}(0)=\bm{x}_{0}.\end{array}$$

$$\displaystyle\begin{array}[]{rll}J^{\star}(\bm{x}(0)):&\stackrel{{\scriptstyle[}}{{\Sigma}}\in\{0,\cdots,2n\}^{N-1}]{}{\text{minimize}}&J_{\Sigma}^{\star}(\bm{x}(0))\;.\end{array}$$

$${\mathcal{C}}_1$$

$${\mathcal{C}}_2$$

$${\mathcal{C}}_3$$

$${\mathcal{C}}_4$$

$$\mathbf{x}(t+1)=\left[\begin{array}{cc}0.9& 0.2\\ 0.8& 1.5\end{array}\right]\mathbf{x}(t)+\left[\begin{array}{c}0.6\\ 0.8\end{array}\right]\mathbf{u}(t),$$

$$\begin{array}[]{ll}\underset{\bm{z}}{\mbox{minimize}}&\dfrac{1}{2}\bm{z}^{\top}F\bm{z}+\mathcal{I}(\bm{w})\\ &G\bm{z}=\bm{h},\\ &\bm{z}=\bm{y},\end{array}$$

$$\mathcal{I}(\bm{y})=\left\{\begin{array}[]{ll}\infty&\mbox{if}\;\eqref{eqn:opt_ADMM_violation}\;\mbox{holds},\\ 0&\mbox{otherwise},\end{array}\right.$$

$$\{\exists t\in \{0,\dots ,N-1\}|\mathbf{y}(t+1)\in {\mathcal{C}}_0\text{mbox}and\mathbf{y}(t+N+2)\ne 0_m\},$$

$$G_{ij}=\left\{\begin{array}[]{ll}A^{i-1}&\;\mbox{for }i=1\dots N,j=1,\\ -I_{n}&\;j=i\;\mbox{and}\;j>1,\\ A^{i+N-j}B&\;\mbox{if}\;j\geq N+2\;\mbox{and}\;i+N\geq j,\\ 0&\;\mbox{otherwise}.\end{array}\right.$$

$${\mathbf{z}}^{k+1/2}$$

$$\displaystyle\left.+\dfrac{\rho}{2}\left\|\left[\begin{array}[]{l}G\\ I\end{array}\right]\bm{z}-\left[\begin{array}[]{l}0\\ I\end{array}\right]\bm{z}^{k}-\left[\begin{array}[]{l}\bm{h}\\ 0\end{array}\right]+\bm{u}^{k}\right\|^{2}\right)$$

$${\mathbf{z}}^{k+1}$$

$${\mathbf{u}}^{k+1}$$

$$\Pi_{i}(\bm{x})=\left\{\begin{array}[]{ll}0&\mbox{if}\;i\geq N+2\mbox{ and }\bm{x}(i-N-1)\in\mathcal{C}_{0}\\ \bm{x}(i)&\mbox{otherwise}.\end{array}\right.$$

$$\begin{gathered} J(\mathbf{x}(t),{\mathbf{\pi }}_t^{\star N-1})=V_f(\mathbf{x}(t+N\mid t)) \\ +\sum _{k=t}^{t+N-1}\ell (\mathbf{x}(k\mid t),\mathbf{u}(k\mid t)) \end{gathered}$$

$$\displaystyle\bm{\pi}_{t}^{\star N-1}=\big{[}\bm{u}^{\star}(t\mid t)^{\intercal},\cdots,\bm{u}^{\star}(t+N-1\mid t)^{\intercal}\big{]}\;,$$

$$\displaystyle\bm{u}(t)=\big{[}I~{}0~{}\cdots~{}0\big{]}\bm{\pi}_{t}^{\star N-1}\;.$$

$$\mu$$

$$\begin{gathered} \mathbf{x}(t+1)=\left[\begin{array}{ccc}0.53& -2.17& 0.62\\ 0.22& -0.06& 0.51\\ -0.92& -1.01& 1.69\end{array}\right]\mathbf{x}(t)+\left[\begin{array}{c}0.4\\ 0.7\\ 0.9\end{array}\right]\mathbf{u}(t) \\ +B_{\mathbf{w}}\mathbf{w}(t). \end{gathered}$$

$${\mathbf{x}}_0={\left[\begin{array}{ccc}\sin \theta \cos \varphi & \sin \theta \sin \varphi & \cos \theta \end{array}\right]}^{⊺},$$

Full text

Index terms** — Optimal control; Linear systems; Event-triggered control; Receding horizon control**

I Introduction

The problem of determining optimal control policies for discrete-time linear systems has been extensively investigated in the literature; see, e.g., [1, 2, 3]. The standard optimal control problem assumes that an unlimited number of control actions is available at the actuator. Although this assumption is valid for many applications, it does not hold for some specific scenarios where communication and computation resources become scarce; for example, (a) control systems with constrained actuation resources [4, 5, 6], (b) control systems with shared processor resources [7], or (c) information exchange over a shared communication channel [8].

To reduce the communication burden between the controller and the actuator, Imer and Başar [9] introduced a constraint on the number of control actions while designing optimal control policies for linear scalar systems. Later, Bommannavar and Başar [10] and Shi et al. [11] extended the work of [9] to a class of higher-order systems. The problem, proposed in [9], can be also formulated as a cardinality-constrained linear-quadratic control problem. The introduction of cardinality constraint changes the standard linear-quadratic control problem from a quadratic programming (QP) to a mixed-integer quadratic programming (MIQP) problem. As it has been proved in [12], it is an NP-complete problem. Therefore, Gao and Lie [13] provided an efficient branch and bound algorithm to compute the optimal control sequence. Since the cardinality constraint is highly non-convex, a convex relaxation of this problem, based on $\ell_{1}$-regularized $\ell_{2}$ optimization, was considered in the literature; see, e.g., [14, 15, 16, 17]. Although $\ell_{1}$-norm regularization of the cost function is an effective way of promoting sparse solutions, no performance guarantees can be provided in terms of the original cost function due to the transformed cost function.

Event- and self-triggered control systems have been broadly used in the literature to reduce the amount of communication between the controller and the actuator while guaranteeing an attainable control performance; see, e.g., [18, 19, 20, 21, 22]. As distinct from sparse control techniques as mentioned earlier, the event- and self-triggered control require the design of both a feedback controller that computes control actions and a triggering mechanism that determines when the control input has to be updated. A vast majority of the works in the literature first designed a controller without considering sparsity constraint, and then, in the subsequent design phase, they developed the triggering mechanism for a fixed controller. In contrast, another line of research concentrates on the design of feedback control law while respecting a predefined triggering condition.

The design of event- and self-triggered algorithms can be extremely beneficial in the context of model predictive control (MPC) strategies; see, e.g., [23, 24, 25, 26, 27, 28, 29, 30, 31]. MPC is a control scheme that solves a finite-horizon optimal control problem at each sampling instant and only applies the first element of the resulting optimal control input trajectories. The use of event-triggered algorithms, therefore, reduces the frequency of solving optimization problems and transmitting control actions from the controller to the actuator, and, consequently, saves computational and communication resources. Lehmann et al. [26] proposed an event-based strategy for discrete-time systems under additive, bounded disturbances. The controller only computes a new control command whenever the difference between actual state and predictive state exceeds a threshold. Sijs et al. [23] combined state estimation with MPC in order to design an event-based estimation framework. The authors of [24, 25] combined MPC with event-triggered sampling strategies based on the ISS concept. As different from the aforecited works, the authors of [19, 20, 21] studied infinite horizon quadratic cost. All works discussed above focus on discrete-time linear/non-linear systems; however, there are also a substantial number of works, which considers continuous-time systems, in the literature; see, e.g., [30, 32, 31]. For instance, the authors of [30, 32] investigated the stability of event-based MPC algorithms for continuous-time nonlinear systems yet they did not consider disturbance. Later, Li and Shi [31] studied the MPC problem for continuous-time nonlinear systems subject to bounded disturbances.

Contributions

In this paper, we formulate a finite-horizon optimal event-triggered control problem where a threshold-based event-triggering algorithm dictates the communication between the controller and the actuator. Then, we propose various algorithms to compute a control action sequence that provides optimal and sub-optimal solutions for this problem, which is, in general, hard to solve due to two main reasons: (a) it has the combinatorial nature since the decisions are binary variable (i.e., transmit or not transmit), and (b) introduction of a threshold-based triggering condition leads to optimizing a convex cost function over a non-convex domain. The main contributions of this paper are threefold:

• (i)

we show that the optimal solution of the control problem, mentioned above, can be determined via solving a set of quadratic programming problems;

• (ii)

we provide an efficient heuristic algorithm based on ADMM to design a control input sequence that provides a sub-optimal solution for the finite-horizon optimal event-triggered control problem;

• (iii)

we describe the receding horizon implementation of the event-triggered control algorithm, including a proof of practical stability.

Outline

The remainder of this paper is organized as follows. Section II formulates the event-triggered finite-horizon LQ control problem and introduces assumptions. Section III provides a simple procedure for designing optimal control laws to minimize the linear-quadratic cost function while Section IV presents two heuristic methods that usually achieves tolerable sub-optimal performance while significantly reducing the computational complexity. In Section V, a receding horizon control scheme with event-triggered algorithm is presented. Section VI demonstrates the effectiveness and advantages of the presented approach while Section VII concludes the paper.

Notation

We write $\mathbb{N}$ for the positive integers, $\mathbb{N}_{0}$ for $\mathbb{N}\cup\{0\}$, and $\mathbb{R}$ for the real numbers. Let $\mathbb{R}_{\succeq 0}^{n}$ denote the set of non-negative real vectors of dimension $n$, and $\mathbb{R}^{n}$ be the set of real vectors of dimension $n$. Vectors are written in bold lower case letters (e.g., $\bm{u}$ and $\bm{v}$) and matrices in capital letters (e.g., $A$ and $B$). If $\bm{u}$ and $\bm{v}$ are two vectors in $\mathbb{R}^{n}$, the notation $\bm{u}\leq\bm{v}$ corresponds to component-wise inequality. The set of all real symmetric positive semi-definite matrices of dimension $n$ is denoted by $\mathbb{S}_{\succeq 0}^{n}$. We let $\mathbf{0}_{n}$ be the $n$–dimensional column vectors of all zeros, $\mathbf{1}_{n}$ be the vectors of all ones. The Kronecker product of two matrices (e.g., $A$ and $B$) is denoted by $A\otimes B$. For any given $\bm{x}\in\mathbb{R}^{n}$, the $\ell_{\infty}$–norm is defined by $\parallel\bm{x}\parallel_{\infty}=\max\limits_{1\leq i\leq n}|x_{i}|$. For a square matrix $A$, $\lambda_{\max}(A)$ denotes its maximum eigenvalue in terms of magnitude. The notation$\{x_{k}\}_{k\in\mathcal{K}}$ stands for $\{x(k):k\in\mathcal{K}\}$, where $\mathcal{K}\subseteq\mathbb{N}_{0}$. The power set of any set $\mathcal{N}$, written $\mathsf{P}(\mathcal{N})$, is the set of all subsets of $\mathcal{N}$, including the empty set and $\mathcal{N}$ itself. The cardinality of a set denoted by $|\mathcal{N}|$.

II Finite Horizon Optimal Event-Triggered Control

We consider the feedback control loop, depicted in Fig. 1. The dynamics of the physical plant $\mathcal{G}$ can be described by the discrete-time linear time-invariant system:

[TABLE]

where $\bm{x}(t)\in\mathbb{R}^{n}$ is the state variable at time instant $t$, $\bm{u}(t)\in\mathbb{R}^{m}$ is the control input at time instant $t$, $A\in\mathbb{R}^{n\times n}$ and $B\in\mathbb{R}^{n\times m}$ are system matrices of appropriate dimensions, and $\bm{x}_{0}\in\mathbb{R}^{n}$ is the given initial condition. The pair $(A,B)$ is assumed to be stabilizable, and $A$ is not necessarily Schur stable.

In this paper, we assume that the sensor $\mathsf{S}$ takes periodic noise-free samples of the plant state $\bm{x}(t)$ and transmits these samples to the controller node. The controller $K$ is event-triggered, and it computes new control commands and transmits them to the actuator $\mathsf{A}$ only at times when $\bm{x}(t)\in\mathcal{S}\triangleq\mathbb{R}^{n}\setminus\mathcal{C}_{0}$ with

[TABLE]

for a given threshold $\varepsilon>0$. In case $\bm{x}(t)\in\mathcal{C}_{0}$, the controller does not compute any new control actions, and the actuator input to the plant is set to zero:

[TABLE]

where

[TABLE]

It is worth noting that $\mathcal{T}$ denotes a set of time instants at which no control computations are needed and, in turn, the plant runs open loop. Notice that $\mathcal{T}$ is not a given set of variables; in contrast, it is generated by the initial state $\bm{x}_{0}$ and the tentative control actions $\bm{u}(t)$ for all $t\in\{0,\cdots,\mathrm{T}-1\}$. Hence, to determine the set $\mathcal{T}$, it is necessary to compute the tentative control actions $\bm{u}(t)$. In other words, the set of time instants $\mathcal{T}$ is strongly coupled to the tentative control inputs $\bm{u}(t)$.

Throughout this paper, we aim at designing an admissible optimal control sequence $\bm{\pi}=\{\bm{u}(0),\bm{u}(1),\cdots,\bm{u}(N-1)\}$ to minimize the quadratic cost function:

[TABLE]

where the matrices $Q$ and $P$ are symmetric and positive semi-definite while $R$ is symmetric and positive definite. Then, we consider the constrained finite-time optimal control problem:

[TABLE]

Recall that the set of time instants, at which no computations (and also transmissions) are needed, i.e., $\mathcal{T}$, is defined in (4).

Remark 1

Note that the event-triggered control problem, described in Section II, is a particular class of hybrid systems. One may, therefore, convert it into an equivalent mixed logical dynamical (MLD) system that represents the system by using a blend of linear and binary constraints on the original variables and some auxiliary variables; see, e.g., [33, 34]. In this paper, instead of using the MLD system formulation, we will exploit geometric properties of the problem at hand, which allows us to propose an optimization problem for optimal finite-horizon control and establish stability result for its receding horizon implementation.

Discussion

Although the problem (11) resembles the cardinality-constrained optimal control problem (proposed in [9, 10, 11, 12, 13, 14, 15, 16, 17]), there is a fundamental difference between them, which stems from the question whether scheduling is an exogenous input or autonomously generated? While solving the cardinality-constrained control problem, one needs to optimize both control action and scheduling sequence, which are strongly coupled to each other in the optimization process. On the other hand, the event-triggered control systems are switched systems with internally forced switchings. In these problems, scheduling sequences are generated implicitly based on the evolution of the state $\bm{x}(t)$ and the control signal $\bm{u}(t)$. The major difficulty of this problem is that scheduling depends on the particular initial condition $\bm{x}_{0}$ and the tentative control input $\bm{u}(t)$, and cannot be explicitly determined unless a specific control signal was given; see the survey paper [35] and references therein.

III Computation of optimal control actions

In this section, we concentrate on finding the control input sequence $\bm{\pi}$ that solves the finite-horizon optimal control problem, proposed in (11). Therefore, we present a framework which is based on dividing the non-convex domain into convex sub-domains. This is a simple yet effective procedure to follow; however, the number of convex optimization problems, which needs to be solved, grows exponentially in the length of the control horizon.

The optimal control problem (11) is hard to solve due to the restriction on the control space. Nevertheless, without loss of generality, it is possible to convert this problem to a set of optimization problems of the form

[TABLE]

where $\mathcal{T}\in\mathsf{P}(\mathcal{N})$ with $\mathcal{N}=\{1,2,\cdots,N-1\}$. To obtain the optimal solution of the problem (11), it is necessary to solve the problem (19) for all subsets of the power set of $\mathcal{N}$, i.e., $\mathsf{P}(\mathcal{N})$, and select the one providing the lowest cost; see Fig. 2. More precisely, this can be written as:

[TABLE]

The number of sub-problems, which are needed to be solved, grows exponentially with the control horizon $N$ (i.e., $|\mathsf{P}(\mathcal{N})|=2^{N-1}$). Bear in mind that since these sub-problems are independent of each other, they can be also solved in parallel. In addition to its combinatorial nature, the problem (19) requires the optimization of a convex function over a non-convex domain. In the literature, there exist some available tools to solve the convex objective non-convex optimization; see, e.g., [36, 37, 38].

As the feasible region $\mathcal{S}$ can be expressed as a finite union of polyhedra, the disjunctive formulation can be applied. In particular, assuming $n$-dimensional problem instance (11), its feasibility region can be divided into $2n+1$ disjunctive polyhedra111See Fig. 3 for an example of two dimensional problem where each disjunctive set is shown in different colors.. Following the construction above, we write:

[TABLE]

where $n$ is the state dimension, and $\mathcal{C}_{p}\subset\mathbb{R}^{n}$ are full-dimensional polyhedra, i.e.,

[TABLE]

for some $T_{p}\in\mathbb{R}^{(2n-1)\times n}$ and $\bm{d}\in\mathbb{R}^{2n-1}$.

We define a piece-wise constant function of time $\sigma(t)$, which takes on values in $\{0,1,\cdots,2n\}$ and whose value $p$ determines, at each time $t\in\{0,\cdots,\mathrm{T}-1\}$, the state variable $\bm{x}(t)$ to belong to the interior of the polyhedron $\mathcal{C}_{p}$. We divide the non-convex set $\mathcal{S}$ into convex subsets $\mathcal{C}_{p}$ with $p\in\{0,1,\cdots,2n\}$ and, at each time step $t\in\{0,\cdots,\mathrm{T}-1\}$, we choose only one active set. The switching signal,

[TABLE]

gives a sequence $\Sigma=\{\sigma(0),\cdots,\sigma(N-1)\}$. Then, for given $\mathcal{T}$ and $\Sigma$, we rewrite the optimization problem (19) as

[TABLE]

It is necessary to solve all possible combination of problem (30) to determine the global optimal solution. Particularly, the problem (11) can be solved as a sequence of $(2n+1)^{N-1}$ quadratic programming (QP) problems. Hence, we have the following series of optimization problems:

[TABLE]

It is worth noting that even though the number of QP sub-problems grows exponentially with $N$ and $n$, all these problems are independent of each other; therefore, one can parallelize the problems and reduce the computation time.

Corollary 1

The global optimal solution of the control problem (11) can be computed by solving a set of convex quadratic programming problems of the form (30), in parallel, and selecting the solution that provides the lowest control cost among all feasible solutions.

The following examples should give the reader a better understanding of the problem.

Example 1

Let $n=2$ and $\mathcal{S}=\mathbb{R}^{2}\setminus\mathcal{C}_{0}$ with $\mathcal{C}_{0}=\{(x_{1},x_{2})\in\mathbb{R}^{2}:-\varepsilon<x_{1}<\varepsilon,-\varepsilon<x_{2}<\varepsilon\}$. The non-convex set $\mathcal{S}$ can be divided into four convex subsets:

[TABLE]

which are illustrated by different colors in Fig. 3. The optimal control inputs can be computed by solving quadratic programs for different combinations of convex sets. In order to compute optimal control inputs, it is necessary to solve $5^{N-1}$ convex quadratic programming problems.

Example 2

Let us consider the second-order system:

[TABLE]

with the initial condition $\bm{x}_{0}=\left[\begin{smallmatrix}0&-1\end{smallmatrix}\right]^{\intercal}$. The event-triggered controller transmits control messages if the $\ell_{\infty}$-norm of the state variable $\bm{x}(t)$ is greater than $\varepsilon=0.25$. The performance indices are given by $Q=\mbox{diag}\{2,2\}$ and $R=5$. The control horizon is chosen as $N=7$. The state trajectories, depicted in Fig. 3, can be obtained via solving the optimization problem (30) for given sets $\Sigma_{1}=\{4,4,4,1,1,0,0\}$, $\Sigma_{2}=\{4,1,1,2,2,0,0\}$, and $\Sigma_{3}=\{4,1,2,2,2,0,0\}$. Fig. 3 shows the state trajectories for two different sets. The set $\Sigma_{1}$ leads to the global optimal solution.

Example 1 demonstrates that obtaining the optimal solution to the finite-horizon optimal control problem, mentioned earlier, requires solving an exponential number of QPs. Therefore, in the next section, we propose heuristic algorithms that significantly reduce computational complexity, by compromising the optimality guarantees slightly.

IV Computation of sub-optimal control actions

In this section, we propose two heuristic algorithms that usually achieve a low control loss while significantly decreasing the computation time. We, firstly, develop a simple algorithm that takes the initial state $\bm{x}_{0}$ as input and finds a suitable sequence $\Sigma$ in a greedy manner by increasing the horizon of the problem from $1$ to $N$. Secondly, we apply an algorithm based on ADMM to compute the heuristic scheduling sequence $\Sigma$. This sequence is then used to solve the disjunctive problem (30) and obtain a good approximate control sequence $\pi$. Both techniques are of polynomial time complexity, hence, reducing the computational efforts significantly. Numerical investigations (see Section VI) indicate that these heuristic algorithms usually provide close to optimal solutions.

IV-A Greedy heuristic method

We construct an intuitive greedy algorithm to solve the optimal control problem, proposed in (11), efficiently. As discussed earlier, given a feasible switching sequence $\Sigma$, a corresponding optimal control problem can be formulated as a quadratic programming problem of the form (30). One, then, solves the combinatorial problem (32) to find the optimal switch sequence $\Sigma^{\star}$, thereby solving the optimal control problem (11).

A good way to deal with the combinatorial nature of this type of algorithms and reduce the computational complexity is to employ a greedy algorithm, which solves a global problem by making a series of locally optimal decisions. Greedy algorithms do not necessarily provide optimal solutions; however, they might determine local optimal solutions, which can be used to estimate a globally optimal solution, in a reasonable time.

Our proposed greedy algorithm works as follows. Since $\bm{x}_{0}$ is known, we can compute corresponding switching signal $\sigma(0)$. Then, we set $\Sigma=\{\sigma(0),p\}$ for all $p\in\{0,1,\dots,2n\}$ and solve $2n+1$ number of QPs of the form (30) to obtain the optimal switch signal $\sigma(1)$. The same procedure then repeats for $k=2,\dots,N-1$, where at each step $k$, we solve $2n+1$ times the QP problem (30) to find the best $\sigma(k)$ keeping the previously found scheduling sequence $\Sigma=\{\sigma(0),\dots,\sigma(k-1)\}$ unchanged.

Algorithm 1 provides the detailed steps of discussed greedy heuristic method. Algorithm 1 executes $(2n+1)(N-1)$ number of QPs compared to $(2n+1)^{N-1}$ as in optimal control procedure. As a result, it runs in polynomial time.

IV-B *A heuristic method based on ADMM *

In this section, we develop a computationally efficient method for solving (11). Our approach is based on, Alternating Direction Method of Multipliers (ADMM), a well developed technique to solve large-scale disciplined problems [39]. The idea of utilizing ADMM as a heuristic to solve non-convex problems has been recently considered in literature [39, 40, 38, 41]. In particular, [38, 40] employ ADMM to approximately solve problems with convex costs and non-convex constraints. The approximate ADMM solutions are improved by multiplicity of random initial starts and a number of local search methods applied to ADMM solutions.

In this paper, using some of the techniques introduced above, we first apply ADMM to original non-convex problem to achieve an intermediate solution. This algorithm, for a general non-convex problem, does not necessarily converge; however, when it does, the corresponding solution gives a good initial guess for finding the optimal solution to this problem. In a second phase of our heuristic method, we perform a polishing technique with the aim of achieving a feasible and hopefully closer to the optimal solution (see [38] for an overview of polishing techniques). Our polishing idea is based on disjunctive programming problem (30) with the fixed event index $\mathcal{T}$ and trajectory sequence $\Sigma$.

Initially developed to solve large-scale convex structured problems, ADMM has been recently advocated, to a great extent, for being effective approximation technique to solve non-convex optimization problems (see [39, 40, 38] and references therein). We start with casting (11) to the ADMM form. Essentially, we have

[TABLE]

where $\bm{z}$ collects the state and control components; i.e., ${\bm{z}\triangleq[\bm{x}(0)^{\top},\dots,\bm{x}(N)^{\top},\bm{u}(0)^{\top},\dots,\bm{u}(N-1)^{\top}]^{\top}}$, $F$ represents the positive definite Hessian of the quadratic cost. That is $F\triangleq\mbox{blkdiag}(Q_{s},R_{s})$ with $Q_{s}\triangleq I_{N+1}\otimes Q$ and $R_{s}\triangleq I_{N}\otimes R$. Moreover, $\mathcal{I}(\bm{y})$ is the indicator function enforcing the event-triggering control law; that is

[TABLE]

[TABLE]

where (36) detects the constraint violation in (11) in terms of having $\bm{x}(t)\in\mathcal{C}_{0}$ while corresponding $\bm{u}(t)\neq\mathbf{0}_{m}$. The constraints in (34) are twofold. The first constraint in (34) describes the LTI system dynamics (1). Here $G\in\mathbb{R}^{nN\times\left(n(N+1)+mN\right)}$ have the following $i=\{1,\dots,N\},\;j=\{1,\dots 2N+1\}$- blocks,

[TABLE]

and $\bm{h}\in\mathbb{R}^{n(N+1)+mN}\triangleq[\bm{x}(0)^{\top},\mathbf{0}_{n}^{\top},\dots,\mathbf{0}_{m}^{\top}]^{\top}$. Finally, the last constraint in (34) is of consensus-type to ensure that non-convex constraint in (11) is asymptotically satisfied.

After formulating the problem, each iteration of ADMM algorithm consists of (see [39] for a complete treatment of the subject):

[TABLE]

Here, $k$ denotes the iteration counter, $\rho>0$ is the step-size (penalty) parameter, and $\Pi$ denotes the projection onto non-convex constraint and is given by

[TABLE]

When the cost function in hand is convex and the constraints set is closed and convex then the ADMM algorithm converges to the optimal solution of the problem. In our case, since $\mathcal{I}(\bm{y})$ is non-convex the ADMM algorithm (37) may not converge to optimal point. Moreover, examples can be found in which (37) does not even converge into a feasible point. To further improve the ADMM solution, we utilize an additional convex problem to polish the intermediate results. That is, we restrict the search space to a convex set that includes the ADMM solution pattern. Let ${\tilde{\bm{z}}\triangleq[\tilde{\bm{x}}(0)^{\top},\dots,\tilde{\bm{x}}(N)^{\top},\tilde{\bm{u}}(0)^{\top},\dots,\tilde{\bm{u}}(N-1)^{\top}]^{\top}}$ be the output of the ADMM algorithm and the sets $\widetilde{\Sigma}$ and ${\widetilde{\mathcal{T}}\triangleq\{t\in\mathbb{N}_{0};t<N:\tilde{\bm{x}}(t)\in\mathcal{C}_{0}\}}$ denote the trajectory sequence associated with $[\tilde{\bm{x}}_{1}^{\top},\dots,\tilde{\bm{x}}_{N}^{\top}]^{\top}$ and the restriction index of $\tilde{\bm{x}}$ into $\mathcal{C}_{0}$, respectively. Then our heuristic polishing procedure formulates (30) with $\mathcal{T}=\mathcal{\widetilde{T}}$ and $\Sigma=\widetilde{\Sigma}$. Algorithm 2 provides a summary of our heuristic method. The input variables include the error-tolerance threshold $\epsilon^{tol}$, randomized initial state $\bm{z}^{0}$ as well as $\bm{z}_{best}$ and $f_{best}$ to store the final solution.

A few comments related to Algorithm 2 are in order.

In contrast to the ADMM algorithms for convex problems that converge to the optimum for all positive range of the step-size parameter $\rho$, the stability of the ADMM algorithm for non-convex problems is sensitive to the choice of $\rho$. Moreover, similar to the convex case, the convergence speed is also affected by the choice of $\rho$. A variety of techniques including proper step-size selection, over relaxation, constraint matrix pre-conditioning and caching can be employed to accelerate the ADMM procedure (37) (see [42, 43] for a reference on the topic). 2. 2.

One can utilize an adaptive iteration count procedure to improve the probability of finding feasible solution to (11). The procedure works as the following. Run the ADMM algorithm for a fixed number of iterations and then check if the disjunctive QP problem has feasible solution. If not, then increase the number of iterations (e.g., double it) and repeat the procedure (run ADMM and disjunctive QP) until finding a feasible solution or reaching to a point where the current ADMM solution does not improve the previous one in terms of constraint violations and quadratic cost. 3. 3.

The variable $z^{0}$ is initialized randomly using a normal distribution. In general, repeating the ADMM algorithm for a multiple of initial points may improve the approximate solution at the cost of extra computational complexity (as suggested in e.g., [40]). However, in our application we did not experience significant performance improvement by repeating the ADMM algorithm (steps $1-6$ in Algorithm 2) with different initializations. One reason could be that in Algorithm 2, we employ the solution trajectories $\widetilde{\Sigma}$ and $\widetilde{\mathcal{T}}$ based on the ADMM solution $\tilde{z}$ and not $\tilde{z}$ itself in the disjunctive problem (30). 4. 4.

Using caching and $LDL^{\top}$ matrix factorization techniques in the first-step of (37) [39], each $\bm{z}$-update in Algorithm 2 costs on the order of $O\left((n(N+1)+mN)^{2}\right)$ for dense $P$ matrices. Furthermore, if we assume $N\geq\max\{n,m\}$ then each ADMM iteration costs on the order of $O(N^{2})$. This means that overall cost of the ADMM algorithm is of $O(KN^{2})$ where $K$ is the number of ADMM iterations. In our application, with proper algorithm parameter selection, the ADMM algorithm converges almost always within a few hundreds of iterations (see the results presented in Section VI). 5. 5.

The computational complexity of the disjunctive step in our heuristic method falls into the generic complexity of convex QPs on the order of $O(\left(|\mathcal{T}|2n+(N-|\mathcal{T}|)(2n-1)\right)^{2})$ [44].

V Receding horizon control

Except a few special cases, finding a solution to an infinite horizon optimal control problem is not possible. Receding horizon control is an alternative scheme to infinite horizon problem that repeatedly requires solving a constrained optimization problem. At each time instant $t\geq 0$, starting at the current state $\bm{x}(t)$ (assuming that a full measurement of the state $\bm{x}(t)$ is available at the current time $t\in\mathbb{N}_{0}$), the following cost function

[TABLE]

subject to system dynamics and constraints involving states and controls is minimized over a finite horizon $N$. Here, the function $\ell$ defines the stage cost, and $V_{f}$ defines the terminal cost. Denote the minimizing control sequence, which is a function of the current state $\bm{x}(t)$, by

[TABLE]

then the control applied to the plant at time $t\geq 0$ is the first element of this sequence, that is,

[TABLE]

Time is then stepped forward one instant, and the procedure described above is repeated for another $N$-step ahead optimization horizon.

It is worth noting that, due to the fundamental limitation of the optimization problem (11), the state $\bm{x}(t)$ never converges to the origin if the system is unstable. It can only converge into a set including the origin. The following result shows the existence of this set:

Theorem 1

Suppose that $\mathcal{D}_{\mu}\triangleq\{\bm{x}\in\mathbb{R}^{n}\,:\,\parallel\bm{x}\parallel_{\infty}\leq\mu\}$ is a neighborhood of the origin, where

[TABLE]

for some $\kappa>0$. Then, the system (1) with event-triggering constraints (3) and (4) is uniformly practically asymptotically stable, i.e., $\displaystyle{\lim_{t\rightarrow\infty}\parallel\bm{x}(t)\parallel_{\infty}\leq\mu}$.

Theorem 1 establishes practical stability of the system (1) with event-triggering constraints (3) and (4). It shows that if provided conditions are met, then the plant state will be ultimately bounded in an $\ell_{\infty}$-norm ball of radius $\mu$. It is worth noting that, as the other stability results, which use Lyapunov techniques, this bound will not be tight. Next, we would like to exemplify the convergence of the state $\bm{x}(t)$ to the set $\mathcal{D}_{\mu}$ from a set of initial conditions $\bm{x}_{0}$.

Example 3

Consider the second-order system, given by (33), with initial conditions $\bm{x}_{0}=1.2\left[\begin{smallmatrix}\text{sin}(\frac{\pi k}{6})~{}\text{cos}(\frac{\pi k}{6})\end{smallmatrix}\right]^{\intercal}$ for all $0\leq k\leq 12$. The controller transmits a control message whenever the event-triggering condition, i.e., $\parallel\bm{x}(t)\parallel_{\infty}>0.25$, holds. The performance indices are chosen as $Q=\mbox{diag}\{2,2\}$ and $R=5$ while the prediction horizon is set to $N=6$. Fig. 4 shows the state trajectories of the receding horizon implementation of the event-triggered control system, described by (1) – (4), for different initial conditions $\bm{x}_{0}$. As can be seen in the same figure, the number of transmitted control commands depends on the initial condition.

VI Numerical examples

VI-A Performance of sub-optimal event-triggered control actions

To illustrate performance benefits of the proposed heuristic algorithms, we consider a third-order plant with the following state-space representation:

[TABLE]

Assume that there is no disturbance acting on the plant, i.e., $B_{\bm{w}}=0$.

Our aim, here, is to minimize (5) with performance indices $Q=P=\mathrm{diag}\{2,2,2\}$ and $R=5$, and the horizon length $N=8$. The initial condition is chosen as

[TABLE]

where $0\leq\theta\leq\pi$ and $0\leq\phi\leq\pi$. To perform simulations, we selected $577$ different data points, which are equidistant over a half spherical surface. We evaluated the performance of the greedy and the ADMM-based heuristic algorithms for all these initial conditions. For each data point on the half sphere, we run $823,543$ number of QPs to obtain the global optimal solution. In total, we run more than $475$ million number of QPs in our evaluation. We used a linux machine with $32$ cores in Intel Core $i7$ Extreme Edition $980X$ to solve these QPs in parallel threads. Within this computing resources, it takes approximately $6$ days to perform simulations for one problem setup.

To compare the (true) optimal control performance with our heuristic method, we performed Algorithm 2 on the same problem set. In particular, we evaluated three sets of problem with $\varepsilon=0.2$, $\varepsilon=0.4$, and ${\varepsilon=0.6}$. It is generally expected that the control problem becomes more challenging with increasing the event-threshold $\varepsilon$. We run the ADMM-based heuristic algorithm with adaptive iteration count procedure described in Section IV-B. Moreover, we picked $\rho=9.8$, $\rho=5.8$, and $\rho=6.9$ respectively, for the aforementioned problem scenarios. Under this setup, in both cases (i.e., $\varepsilon=0.2$ and $\varepsilon=0.4$), the ADMM method produced always feasible solutions for maximum $300$ number of ADMM-iterations. However, when $\varepsilon=0.6$ and $\rho=6.9$, the method created infeasible solutions for two initial conditions: $\bm{x}_{0}=\left[0.3036,\,0.2330,\,0.9239\right]^{\top}$ and $\bm{x}_{0}=\left[0.6830,\,0.1830,\,0.7071\right]^{\top}$. To compute feasible solutions, we needed to adjust $\rho$ accordingly for these initial conditions.

Fig. 5 shows the outcome of the comparison. As it shows, for $\varepsilon=0.2$, the ADMM-based heuristic algorithm is able to find near optimal solutions (with relative error less than $5\%$) in almost all cases, and for $\varepsilon=0.4$ and $\varepsilon=0.6$, the algorithm finds solutions with relative error less than $5\%$ in more than $80\%$ of problem instances. The average optimality gap $(J_{\text{admm}}-J^{\star})/J^{\star}$ for the three cases were $0.0035$, $0.0237$, and $0.0716$, respectively. Finally, by comparing the cardinality of control event index $\mathcal{T}$ for the optimal solutions and our ADMM method, one concludes that the heuristic method in more than $50\%$ of cases finds similar-sparse solution as the optimum ones, and in the rest of cases it tends to find more sparse control actions.

We also compared the true optimal control performance with a greedy heuristic method, shown in Algorithm 1, for three sets of problems, mentioned above, with $\varepsilon=0.2$, $\varepsilon=0.4$, and ${\varepsilon=0.6}$. The greedy algorithm detected $22$, $90$, and $135$ optimal solutions of the aforementioned $577$ problems, respectively. Also, it computed the optimal solution with relative error less than $5\%$ in more than $65\%$ of problems at hand for all three problem setups. It is important to note that the greedy heuristic method always provided feasible solutions for all problem cases. The average optimality gaps $(J_{\text{greedy}}-J^{\star})/J^{\star}$ in all three cases were $0.0514$, $0.0780$, and $0.0826$, respectively. As compared to the ADMM-based heuristic method, the optimality gaps became larger.

VI-B Receding horizon control implementation

We consider the open-loop unstable discrete-time system represented by (40) with $B_{\bm{w}}=0$. The prediction horizon is chosen to be $N=6$ with performance indices $Q=P=\mathrm{diag}\{2,2,2\}$ and $R=5$. The event-threshold $\varepsilon$ for the event-triggered receding horizon implementation is set to $0.4$. Fig. 7 demonstrates the time responses of the event-triggered receding horizon control system for the initial condition $\bm{x}_{0}=\left[0,\,\frac{\sqrt{2}}{2},\,-\frac{\sqrt{2}}{2}\right]^{\top}$. Here, two different approaches for computing control input trajectories are considered: the exhaustive search algorithm and the heuristic ADMM algorithm with $\rho=4.8$. As can be seen in Fig. 7, while the state trajectories associated to two control approaches deviate from one another (except for the few first sampling instants that they match together), the corresponding control inputs follow a rather similar sparsity pattern. The event-triggered receding horizon control via exhaustive search algorithm uses the communication channel $14$ times over the $50$ sampling instants, which results in a reduction of $72\%$ in communication and computational burden, whereas the event-triggered receding horizon control via ADMM algorithm uses the channel $16$ times over the $50$ sampling instants, which leads to a reduction of $68\%$ in communication and computational burden. For the aforementioned initial value, the exhaustive search achieves a better control cost: $65.42$ compared to $77.72$ as of the control cost of the ADMM-based heuristic. However, this conclusion does not hold in general. In particular, we experienced cases where starting from a different initial value, the receding horizon controller derived by exhaustive search performs worse than the one based on ADMM-based heuristic.

VI-C Trade-off between communication rate and control performance

We consider the case where a random disturbance $\bm{w}(t)\in\mathbb{R}^{n}$, which is modeled by a discrete-time zero-mean Gaussian white process with co-variance $\Sigma_{\bm{w}}$, is acting on the plant characterized by the injection matrix $B_{\bm{w}}=\mbox{diag}\{1,1,1\}$. The initial condition $\bm{x}_{0}$ is modeled as a random variable having a normal distribution with zero mean and co-variance $\Sigma_{\bm{x}_{0}}$. The process noise $\bm{w}(t)$ is independent of the initial condition $\bm{x}_{0}$.

The control performance is measured by a quadratic function:

[TABLE]

while the communication rate is measured by

[TABLE]

The visualization of the trade-off between the control loss and the communication rate is demonstrated in Fig. 8. Different data points on the curve correspond to different event-threshold values ranging from $0.5$ to $4.0$, and the control loss (41) and the communication rate (42) are evaluated via Monte Carlo simulations. Note that the communication rate decreases dramatically with an increased control loss as the threshold $\varepsilon$ varies between [math] and $2.25$ (dark colors). For $\varepsilon>2.25$ (lighter color), both quantities become less sensitive to changes in the threshold value. We can also identify $0\leq\varepsilon\leq 1.5$ as a particularly attractive region where a large decrease in the communication rate can be obtained for a small loss in control performance.

VII Conclusion

In this paper, we considered a feedback control system where an event-triggering rule dictates the communication between the controller and the actuator. We investigated the optimal control problem with an additional constraint on the event-triggered communication. While this optimization problem, in general, is non-convex, we used a disjunctive programming formulation to obtain the optimal solution via solving an exponential number of quadratic programs. Then, we proposed a heuristic algorithms to reduce the computational efforts while achieving an acceptable performance. Later, we provided a complete stability analysis of receding horizon control that uses a finite-horizon optimization in the proposed class. Our numerical study confirmed the theory.

VIII Appendix

We begin with the definition of the practical stability of control systems.

Definition 1** **(practical stability [45])

The system (1) is said to be uniformly practically asymptotically stable in $\mathcal{A}\subseteq\mathbb{R}^{n}$ if $\mathcal{A}$ is a positively invariant set for (1) and if there exist a $\mathcal{KL}$-function $\beta$, and a nonnegative constant $\delta\geq 0$ such that

[TABLE]

Definition 2** **(practical-Lyapunov function [45])

A function $V:\mathbb{R}^{n}\rightarrow\mathbb{R}_{\geq 0}$ is said to be a practical-Lyapunov function in $\mathcal{A}$ for the system (1) if $\mathcal{A}$ is a positively invariant set and if there exist a compact set, $\Omega\subseteq\mathcal{A}$, neighbourhood of the origin, some $\mathcal{K}_{\infty}$-functions $\alpha_{1},\alpha_{2}$ and $\alpha_{3}$, and some constants $d_{1},d_{2}\geq 0$, such that

[TABLE]

Proof of Theorem 1. To prove the practical stability of the system (1) with a set of control actions $\bm{\pi}^{\star}=\{\bm{u}^{\star}(t\mid t),\cdots,\bm{u}^{\star}(t+N-1\mid t)\}$, which fulfills constraints (3) and (4), we will analyze the value function,

[TABLE]

where $J(\bm{x}(t),\bm{\pi})$ is as seen in (5). We borrow the shifted sequence approach, described in [46], and we use a feasible control sequence, i.e., $\tilde{\bm{\pi}}=\{\bm{u}^{\star}(t+1\mid t),\cdots,\bm{u}^{\star}(t+N-1\mid t),\hat{\bm{u}}\}$. By the optimality property, we obtain the following bound (with $\bm{x}(t\mid t)=\bm{x}(t)$ and $\bm{u}^{\star}(t\mid t)=\bm{u}(t)$):

[TABLE]

for all $\bm{x}(t)\in\mathbb{R}^{n}$. We now investigate the quantity of

[TABLE]

where \Delta V_{f}(t)\triangleq V_{f}\big{(}A\bm{x}(t+N\mid t)+B\hat{\bm{u}}\big{)}-V_{f}(\bm{x}(t+N\mid t)). Based on $\bm{x}(t+N\mid t)$, there are two possibilities:

• •

Suppose that $\parallel\bm{x}(t+N\mid t)\parallel_{\infty}>\varepsilon$, then for any feasible control law, i.e., $\hat{\bm{u}}=K\bm{x}(t+N\mid t)$, the following inequality holds:

[TABLE]

where $A_{K}=A+BK$ (i.e., Schur) and $Q^{*}=Q+K^{\intercal}RK$. From (46) and (• ‣ VIII), it follows that

[TABLE]

• •

Suppose that $\parallel\bm{x}(t+N\mid t)\parallel_{\infty}\leq\varepsilon$, then $\hat{\bm{u}}=0$. Hence, it follows that:

[TABLE]

where we used the norm inequality $\parallel\bm{v}\parallel_{2}\leq\sqrt{n}\parallel\bm{v}\parallel_{\infty}$ for any $\bm{v}\in\mathbb{R}^{n}$; see [47].

As a result, we conclude that, for any $\bm{x}(t+N\mid t)\in\mathbb{R}^{n}$, the following inequality holds:

[TABLE]

where \eta=\lambda_{\max}\big{(}A^{\intercal}PA+Q\big{)}n\varepsilon{}^{2}. Inserting (48) and $a_{1}\parallel\bm{x}(t)\parallel_{2}^{2}\leq\ell(\bm{x}(t),\bm{u}(t))$ where $a_{1}\triangleq\lambda_{\min}(Q)$ into (46) yields:

[TABLE]

Now, let the binary variable $\delta(t)\in\{0,1\}$, for each time instant $t$, represent the transmission of the control action as follows:

[TABLE]

For any feasible control sequence, i.e., $\bm{\pi}^{\prime}=\{K\bm{x}(t),\cdots,K\bm{x}(t+N-1\mid t)\}$, there exists an associated scheduling sequence, denoted by $\Delta(t)=\{\delta(t),\cdots,\delta(t+N-1)\}$. Applying these feasible control inputs, the upper bound of $V_{N}^{\star}(\bm{x})$ becomes:

[TABLE]

where $a_{3}\triangleq\displaystyle{\max_{\Delta\in\mathcal{S}}\lambda_{\max}(S_{\Delta})}$ and

[TABLE]

with $Q_{\delta(i)}\triangleq\delta(i)Q^{*}+(1-\delta(i))Q$ and $A_{\delta(i)}\triangleq\delta(i)A_{K}+(1-\delta(i))A$ for all $i\in\{0,\cdots,N-1\}$. Here, $\mathcal{S}$ denotes a set of all possible feasible schedules generated by a stabilizing control sequence $\bm{\pi}^{\prime}$.

To obtain the lower bound, we have $\bm{x}^{\intercal}(t)Q\bm{x}(t)\leq V_{N}^{\star}(\bm{x}(t))$ and hence $a_{2}\parallel\bm{x}(t)\parallel_{2}^{2}\leq V_{N}^{\star}(\bm{x}(t))$ where $a_{2}\triangleq\lambda_{\min}(Q)$. From, (49) and (VIII) we establish the following relation:

[TABLE]

which implies that

[TABLE]

Since $0<a_{2}\leq a_{3}$, it follows that $\gamma=1-\frac{a_{2}}{a_{3}}$, i.e., $0<\gamma\leq 1$. Therefore, by iterating (53), it is possible to exponentially bound the state evolution via:

[TABLE]

Using the norm inequality $\parallel\bm{v}\parallel_{\infty}\leq\parallel\bm{v}\parallel_{2}\leq\sqrt{n}\parallel\bm{v}\parallel_{\infty}$ for any $\bm{v}\in\mathbb{R}^{n}$ (see [47]), we get:

[TABLE]

Thus, $\displaystyle{\lim_{t\rightarrow\infty}\parallel\bm{x}(t)\parallel_{\infty}\leq\mu}$. $\hfill\blacksquare$