Learning Model Predictive Control for Iterative Tasks: A Computationally Efficient Approach for Linear System

TL;DR

This paper introduces an efficient Learning Model Predictive Control method for linear systems that learns from previous iterations to improve performance while reducing computational complexity.

Contribution

It extends existing LMPC frameworks with a new approach that simplifies computation and guarantees stability using convex safe sets and terminal costs.

Findings

Effective in reducing computational load

Ensures recursive feasibility and performance improvement

Validated through simulation results

Abstract

A Learning Model Predictive Controller (LMPC) for linear system in presented. The proposed controller is an extension of the LMPC [1] and it aims to decrease the computational burden. The control scheme is reference-free and is able to improve its performance by learning from previous iterations. A convex safe set and a terminal cost function are used in order to guarantee recursive feasibility and non-increasing performance at each iteration. The paper presents the control design approach, and shows how to recursively construct the convex terminal set and the terminal cost from state and input trajectories of previous iterations. Simulation results show the effectiveness of the proposed control logic.

Tables

Table 1. Table 1 : Optimal cost of the LMCPC at each j 𝑗 j -th iteration

Iteration     Iteration Cost
	$j=0$	$57.1959612235$
	$j=1$	$49.9313760793$
	$j=2$	$49.9166091658$
	$j=3$	$49.9163668042$
	$j=4$	$49.9163602472$
	$j=5$	$49.9163600537$
	$j=6$	$49.9163600469$
	$j=7$	$49.9163600468$
	$j=8$	$49.9163600464$

Table 2. Table 2: Convergence of the LMPC for different initial conditions

$x_S$	$\overline{N}$	$\overline{\sigma }$	$\Delta J$	Iterations
${[-3.95,-0.05]}^T$	2	$2.6\times {10}^{-1}$	$1.3\times {10}^{-1}$	$44$
${[-3.95,-0.05]}^T$	3	$1.9\times {10}^{-5}$	$1.7\times {10}^{-17}$	$26$
${[-3.95,-0.05]}^T$	4	$8.6\times {10}^{-6}$	$1.8\times {10}^{-20}$	$8$
${[-4,0]}^T$	2	$3.6\times {10}^{-1}$	$4.2\times {10}^{-1}$	$74$
${[-4,0]}^T$	3	$1.6\times {10}^{-5}$	$5.9\times {10}^{-18}$	$26$
${[-4,0]}^T$	4	$5.2\times {10}^{-6}$	$1.2\times {10}^{-20}$	$8$
${[-2,2]}^T$	3	$7.8\times {10}^{-2}$	$5.1\times {10}^{-3}$	$80$
${[-2,2]}^T$	3	$1.7\times {10}^{-5}$	$3.4\times {10}^{-17}$	$22$
${[-2,2]}^T$	4	$7.3\times {10}^{-6}$	$5.7\times {10}^{-20}$	$8$
${[0,1.5]}^T$	2	$1.0\times {10}^{-1}$	$1.5\times {10}^{-2}$	$45$
${[0,1.5]}^T$	3	$1.8\times {10}^{-5}$	$3.4\times {10}^{-17}$	$27$
${[0,1.5]}^T$	4	$6.6\times {10}^{-6}$	$2.0\times {10}^{-19}$	$8$

Equations

$$x_{t+1}=A{x}_t+B{u}_t,$$

$$x_t\in \mathcal{X},u_t\in \mathcal{U},\forall t\in {\mathbb{Z}}_{0+}.$$

$${\mathbf{u}}^j=[u_0^j,u_1^j,...,u_t^j,...],$$

$${\mathbf{x}}^j=[x_0^j,x_1^j,...,x_t^j,...],$$

$$x_0^j=x_S,\forall j\ge 0.$$

$$J_{0\to \infty }^{\ast }(x_S)$$

$$x_{k+1}=A{x}_k+B{u}_k,\forall k\ge 0$$

$$x_0=x_S,$$

$$x_k\in \mathcal{X},u_k\in \mathcal{U},\forall k\ge 0$$

$$h(x_F,0)=0\text{and}h(x_t^j,u_t^j)\succ 0\forall x_t^j\in$$

$$u_t^j\in {\mathbb{R}}^m\setminus \{0\},$$

$$x_F=A{x}_F.$$

$${\mathcal{SS}}^j=\left\{\bigcup _{i\in M^j}\bigcup _{t=0}^{\infty }{x}_t^i\right\}.$$

$$\displaystyle M^{j}=\textrm{}\Big{\{}k\in[0,j]:\lim_{t\to\infty}x_{t}^{k}=x_{F}\Big{\}}.$$

$$\displaystyle\mathcal{CS}^{j}=\text{Conv}(\mathcal{SS}^{j})=\Big{\{}\sum_{i=1}^{|\mathcal{SS}^{j}|}\alpha_{i}z_{i}:\alpha_{i}\geq 0,\sum_{i=1}^{|\mathcal{SS}^{j}|}\alpha_{i}=1,$$

$$\displaystyle z_{i}\in\mathcal{SS}^{j}\Big{\}},$$

$${\mathcal{CS}}^i\subseteq {\mathcal{CS}}^j,\forall i\le j.$$

$$J_{t\to \infty }^j(x_t^j)=\sum _{k=0}^{\infty }h(x_{t+k}^j,u_{t+k}^j),$$

$$J_{0\to \infty }^j(x_0^j)=\sum _{k=0}^{\infty }h(x_k^j,u_k^j).$$

$$P^j(x)=\{\begin{array}{ll}{p}^{j,\ast }(x)& \text{mbox}Ifx\in {\mathcal{CS}}^j\\ +\infty & \text{mbox}Ifx\notin {\mathcal{CS}}^j\end{array}$$

$$p^{j,\ast }(x)$$

$$s.t.$$

$$\sum _{k=0}^j\sum _{t=0}^{\infty }{\lambda }_t^k=1$$

$$\sum _{k=0}^j\sum _{t=0}^{\infty }{\lambda }_t^k{x}_t^k=x,$$

$$P^j(x)=$$

$$=$$

$$p^{j,\ast }(x)$$

$$s.t.$$

$$\sum _{k=0}^j\sum _{t=0}^{t_j}{\lambda }_t^k=1$$

$$\sum _{k=0}^j\sum _{t=0}^{t_j}{\lambda }_t^k{x}_t^k=x.$$

$$J_{t\to t+N}^{\text{scalebox}0.4LMPC,j}$$

$$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}+P^{j-1}(x_{t+N|t})\bigg{]}$$

$$x_{k+1|t}=A{x}_{k|t}+B{u}_{k|t},\forall k\in [t,\cdots ,t+N-1]$$

$$x_{t|t}=x_t^j,$$

$$x_{k|t}\in \mathcal{X},u_k\in \mathcal{U},\forall k\in [t,\cdots ,t+N-1]$$

$$x_{t+N|t}\in {\mathcal{CS}}^{j-1},$$

$${\mathbf{u}}_{t:t+N|t}^{\ast ,j}$$

$${\mathbf{x}}_{t:t+N|t}^{\ast ,j}$$

$$u_t^j=u_{t|t}^{\ast ,j}.$$

$$\begin{array}{c}[x_0^0,x_1^0,...,x_N^0]\in {\mathcal{CS}}^{j-1},\end{array}$$

Full text

1 INTRODUCTION

Iterative Learning Control (ILC) studies control design for autonomous systems performing repetitive tasks Bristow et al. (2006); Lee and Lee (2007); Wang et al. (2009). One task execution is often referred to as “iteration” or “trial”. In ILC, at each iteration, the system starts from the same initial condition and the controller objective is to track a given reference, rejecting periodic disturbances Bristow et al. (2006); Lee and Lee (2007). The tracking error from the previous iterations is used to improve the tracking performance of the closed loop system. Different strategies have been proposed to guarantee zero tracking error of the closed loop system Bristow et al. (2006); Lee and Lee (2007); Wang et al. (2009).

Several control frameworks which combine ILC and MPC strategies have been proposed in literature, Subbaraman and Benosman (2016); Lee and Lee (2000); Lee et al. (2000). In the classical ILC approach the goal of the controller is to track a reference trajectory, however, in some application such has autonomous racing Sharp and Peng (2011); Rucco et al. (2015) or for some manipulation tasks Tamar et al. (2016), it may be challenging to generate a priori a reference trajectory that maximize the system performance. For this reason, a very recent work Tamar et al. (2016) proposed a reference-free ILC scheme. The authors used a MPC controller with a terminal cost that allows to consider the long term planning. This terminal cost is computed using a neural network trained on data generated by offline simulations. The authors were able to improve the system performance over iterations. However, no guaranties about stability, recursive feasibility and performance improvement are provided.

Our objective is to design a reference-free iterative control strategy for linear system able to learn from previous iterations. At each iteration, the initial condition, the constraints and the objective function do not change. The $j$-th iteration cost is defined as the objective function evaluated for the realized closed loop system trajectory. The iteration cost shall not increase over the iterations and state and input constraints shall be satisfied. Model Predictive Control is an appealing technique to tackle this problem for its ability to handle state and inputs constraints while minimizing a finite-time predicted cost Garcia et al. (1989). However, the receding horizon nature can lead to infeasibility and it does not guaranty improved performance at each iteration Mayne et al. (2000).

The contribution of this paper is the following. We present an extension to the learning MPC for iterative control task in Rosolia and Borrelli (2017). In particular, we introduce a new formulation for linear system that drastically reduces the computation burden of the controller without compromising the guaranties of the learning MPC. We show how to design a convex safe set and a terminal cost function in order to guarantee: (i): [asymptotic stability], the closed loop system converges asymptotically to the equilibrium point, (ii): [persistent feasibility], state and input constraints are satisfied if they were satisfied at iterations $j-1$ (iii): [performance improvement], the $j$-th iteration cost does not increase compared with the $j$-$1$-th iteration cost, (iv): [global optimality], if the steady state system converges to a closed-loop trajectory as the number of iterations $j$ goes to infinity, then that closed-loop trajectory is globally optimal. We emphasize that (i)-(ii) are standard MPC design requirement and (iii)-(iv) are the core contribution of this work.

This paper is organized as follows: in Section II we introduce the notation used throughout the paper. Then, we define the convex safe set and the terminal cost function used in the design of the learning MPC. Section III describes the control design. We show the recursive feasibility and stability of the control logic and, afterwards, we prove the convergence properties. Finally, in Section IV we test the proposed control logic on an infinite horizon linear quadratic regulator and we compare the computational efficiency with the learning MPC from Rosolia and Borrelli (2017).

2 PROBLEM FORMULATION

Consider the discrete time system

[TABLE]

where $x\in{\mathbb{R}}^{n}$ and $u\in{\mathbb{R}}^{m}$ are the system state and input, respectively, subject to the constraints

[TABLE]

where $\mathcal{X}$ and $\mathcal{U}$ are convex sets.

At the $j$-th iteration the vectors

[TABLE]

collect the inputs applied to system (1) and the corresponding state evolution. In (3), $x_{t}^{j}$ and $u_{t}^{j}$ denote the system state and the control input at time $t$ of the $j$-th iteration, respectively. We assume that at each $j$-th iteration the closed loop trajectories start from the same initial state,

[TABLE]

The goal is to design a controller which solves the following infinite horizon optimal control problem at each iteration:

[TABLE]

where equations (5b) and (5c) represent the system dynamics and the initial condition, and (5d) are the state and input constraints. The stage cost, $h(\cdot,\cdot)$, in equation (5a) is continuous, jointly convex and it satisfies

[TABLE]

where the final state $x_{F}$ is assumed to be a feasible equilibrium for the unforced system (1)

[TABLE]

Next we introduce the definition of the convex safe set and of the terminal cost. Both will be used later to guarantee stability and feasibility of the learning MPC for linear system.

2.1 Convex Safe Set

In the following we recall the definition of the sampled Safe Set from Rosolia and Borrelli (2017) which is necessary to construct the convex Safe Set used in the learning MPC for linear system.

The definition of the sampled Safe Set exploits the iterative nature of the control task to define an invariant control set, using the realized system trajectories. At the $j$-th iteration the sampled safe set, $\mathcal{SS}^{j}$, is defined as

[TABLE]

$\mathcal{SS}^{j}$ is the collection of all state trajectories at iteration $i$ for $i\in M^{j}$. $M^{j}$ in equation (8) is the set of indexes $k$ associated with successful iterations $k$ for $k\leq j$, defined as:

[TABLE]

Moreover, as $\mathcal{X}$ and $\mathcal{U}$ are convex, for each convex combination of the elements in $\mathcal{SS}^{j}$ we can find a control sequence that steers the system (1) to $x_{F}$. Therefore, the convex Safe Set, defined as

[TABLE]

is a control invariant set. Note that $|\mathcal{SS}^{j}|$ is the cardinality of $\mathcal{SS}^{j}$. For further details on control invariant set we refer to Borrelli (2003).

From (9) we have that $M^{i}\subseteq M^{j},\forall i\leq j$, which implies that

[TABLE]

2.2 Terminal Cost

At time $t$ of the $j$-th iteration the cost-to-go associated with the closed loop trajectory (3b) and input sequence (3a) is defined as

[TABLE]

where $h(\cdot,\cdot)$ is the stage cost of problem (5). We define the $j$-th iteration cost as the cost (12) of the $j$-th trajectory at time $t=0$,

[TABLE]

$J_{0\rightarrow\infty}^{j}(x_{0}^{j})$ quantifies the controller performance at each $j$-th iteration.

Remark 1

In equations (12)-(13), $x_{k}^{j}$ and $u_{k}^{j}$ are the realized state and input at the $j$-th iteration, as defined in (3).

Finally we define the, barycentric function (Jones and Morari (2010))

[TABLE]

where

[TABLE]

where $x_{t}^{k}$ is the realized state at time $t$ of the $j$-th iteration, as defined in (3b).

Remark 2

The function $P^{j}(x)$ assigns to every point in $\mathcal{CS}^{j}$ the minimum cost-to-go along the trajectories in $\mathcal{CS}^{j}$, in particular we have that $\forall x\in\mathcal{CS}^{j},$

[TABLE]

*where $\lambda^{*,k}_{t}$ is the minimizer in (15).

Remark 3

In practical applications each $j$-th iteration has a finite time duration $t_{j}$, and therefore $p^{j,*}(x)$ is reformulated as

[TABLE]

3 LMPC FOR LINEAR SYSTEM

In this section we present the design of the proposed Learning Model Predictive Control (LMPC). We first assume that there exists a feasible input sequence that steers the system from the initial point $x_{S}$ to terminal point $x_{F}$ at the [math]-th iteration. Then we prove that the proposed LMPC is guaranteed to be recursively feasible, i.e. feasible at all time instants of every successive iteration. Moreover, we show that the LMPC guaranties a non-increasing iterations cost between two successive executions of the task.

3.1 LMPC Control Design

The LMPC tries to compute a solution to the infinite time optimal control problem (5) by solving at time $t$ of iteration $j$ the finite time constrained optimal control problem

[TABLE]

where (18b) and (18c) represent the system dynamics and initial condition, respectively. The state and input constraints are given by (18d). Finally (18e) forces the terminal state into the set $\mathcal{CS}^{j-1}$ defined in equation (10).

Let

[TABLE]

be the optimal solution of (18) at time $t$ of the $j$-th iteration and $J_{t\rightarrow t+N}^{\scalebox{0.4}{LMPC},j}(x_{t}^{j})$ the corresponding optimal cost. Then, at time $t$ of the iteration $j$, the first element of ${\bf{u}}^{*,j}_{t:t+N|t}$ is applied to the system (1)

[TABLE]

The finite time optimal control problem (18) is repeated at time $t+1$, based on the new state $x_{t+1|t+1}=x_{t+1}^{j}$ (18c), yielding a moving or receding horizon control strategy.

Remark 4

Problem (18) is a convex optimization problem as the terminal constraint (18a) enforces the terminal state in the convex set $\mathcal{CS}^{j-1}$ and the terminal cost $P^{j-1}(\cdot)$ in (18e) is a convex function. This new formulation of the LMPC (18), (20) as a convex problem is the main contribution of this work compared to Rosolia and Borrelli (2017).

Assumption 1

At iteration $j=1$ we assume that $\mathcal{CS}^{j-1}=\mathcal{CS}^{0}$ is a non-empty set and that the trajectory ${\bf{x}}^{0}\in\mathcal{CS}^{0}$ is feasible and convergent to $x_{F}$.

In the next section we prove that, under Assumption 1, the LMPC (18) and (20) in closed loop with system (1) guarantees recursively feasibility and stability, and non-increase of the iteration cost at each iteration.

3.2 Recursive feasibility and stability

In this Section, the properties of $\mathcal{CS}^{j}$ and $P^{j}(\cdot)$ are used to show recursive feasibility and asymptotic stability of the equilibrium point $x_{F}$.

Theorem 1

Consider system (1) controlled by the LMPC controller (18) and (20). Let $\mathcal{CS}^{j}$ be the convex safe set at iteration $j$ as defined in (10). Let assumption 1 hold, then the LMPC (18) and (20) is feasible $\forall~{}t\in{\mathbb{Z}}_{0+}$ and iteration $j\geq 1$. Moreover, the equilibrium point $x_{F}$ is asymptotically stable for the closed loop system (1) and (20) at every iteration $j\geq 1$.

Proof: The proof follows from standard MPC arguments. By assumption $\mathcal{CS}^{0}$ is non empty. From (11) we have that $\mathcal{CS}^{0}\subseteq\mathcal{CS}^{j-1}~{}\forall j\geq 1$, and consequently $\mathcal{CS}^{j-1}$ is a non empty set. In particular, there exists a trajectory ${\bf{x}}^{0}\in\mathcal{CS}^{0}\subseteq\mathcal{CS}^{j-1}$. From (4) we know that $x_{0}^{j}=x_{S}~{}\forall j\geq 0$. At time $t=0$ of the $j$-th iteration the $N$ steps trajectory

[TABLE]

and the related input sequence,

[TABLE]

satisfy input and state constrains (18b)-(18c)-(18d). Therefore (21)-(22) is a feasible solution to the LMPC (18) and (20) at $t=0$ of the $j$-th iteration.

Assume that at time $t$ of the $j$-th iteration the LMPC (18) and (20) is feasible and let ${\bf{x}}^{*,j}_{t:t+N|t}$ and ${\bf{u}}^{*,j}_{t:t+N|t}$ be the optimal trajectory and input sequence, as defined in (19). From (18c) and (20) the realized state and input at time $t$ of the $j$-th iteration are given by

[TABLE]

Moreover, the terminal constraint (18e) enforces $x^{*,j}_{t+N|t}\in\mathcal{CS}^{j-1}$ and, from (15) and (18a),

[TABLE]

We define

[TABLE]

and

[TABLE]

Since the state update in (1) and (18b) are assumed identical we have that

[TABLE]

At time $t+1$ of the $j$-th iteration the input sequence and the related feasible state trajectory

[TABLE]

satisfy input and state constrains (18b)-(18c)-(18d). Therefore, (28) is a feasible solution for the LMPC (18) and (20) at time $t+1$.

We showed that at the $j$-th iteration, $\forall j\geq 1$ , (i): the LMPC is feasible at time $t=0$ and (ii): if the LMPC is feasible at time $t$, then the LMPC is feasible at time $t+1$. Thus, we conclude by induction that the LMPC in (18) and (20) is feasible $\forall j\geq 1$ and $t\in{\mathbb{Z}}_{0+}$.

Next we use the fact the Problem (18) is time-invariant at each iteration $j$ and we replace $J_{t\rightarrow t+N}^{\scalebox{0.4}{LMPC},j}(\cdot)$ with $J_{0\rightarrow N}^{\scalebox{0.4}{LMPC},j}(\cdot)$. In order to show the asymptotic stability of $x_{F}$ we have to show that the optimal cost, $J_{0\rightarrow N}^{\scalebox{0.4}{LMPC},j}(\cdot)$, is a Lyapunov function for the equilibrium point $x_{F}$ (7) of the closed loop system (1) and (20) Borrelli (2003). Continuity of $J_{0\rightarrow N}^{\scalebox{0.4}{LMPC},j}(\cdot)$ can be shown as in Mayne et al. (2000). Moreover from (5a), $J_{0\rightarrow N}^{\scalebox{0.4}{LMPC},j}(x)\succ 0~{}\forall~{}x\in{\mathbb{R}}^{n}\setminus\{x_{F}\}$ and $J_{0\rightarrow N}^{\scalebox{0.4}{LMPC},j}(x_{F})=0$. Thus, we need to show that $J_{0\rightarrow N}^{\scalebox{0.4}{LMPC},j}(\cdot)$ is decreasing along the closed loop trajectory.

From (27) we have $x_{t+1|t}^{*,j}=x_{t+1}^{j}$, which implies that

[TABLE]

Given the optimal input sequence and the related optimal trajectory in (19) and the definition of the $P^{j-1}(\cdot)$ (16), the optimal cost is given by

[TABLE]

We can further simplify the above expression using (15c), (24)-(26) and the fact that $h(\cdot,\cdot)$ is jointly convex in the arguments,

[TABLE]

Note that, in the above derivation, we used the fact that $\bar{\lambda}_{0}^{k}=0$ and $\bar{\lambda}_{t+1}^{k}=\lambda_{t}^{*,k},~{}\forall k\in\{0,j-1\},~{}t\in{\mathbb{Z}}_{0+}$ is a feasible solution to problem (15) and therefore $\sum_{t=0}^{\infty}\lambda^{*,k}_{t}J_{t\rightarrow\infty}^{k}(x_{t+1}^{k})$ is a upper bound for $P^{j-1}(\bar{x})$. Finally, from equations (20), (23) and (29)-(31) we conclude that the optimal cost is a decreasing Lyapunov function along the closed loop trajectory,

[TABLE]

Equation (32), the positive definitiveness of $h(\cdot,\cdot)$ and the continuity of $J_{0\rightarrow N}^{\scalebox{0.4}{LMPC},j}(\cdot)$ imply that $x_{F}$ is asymptotically stable. $\blacksquare$

3.3 Convergence properties

In this Section we assume that the LMPC (18) and (20) converges to a steady state trajectory. We show two results. First, the $j$-th iteration cost $J_{0\rightarrow\infty}^{j}(\cdot)$ does not worsen as $j$ increases. Second, the steady state trajectory is the solution to the infinite horizon control problem (5).

Theorem 2

Consider system (1) in closed loop with the LMPC controller (18) and (20). Let $\mathcal{CS}^{j}$ be the convex safe set at the $j$-th iteration as defined in (10). Let assumption 1 hold, then the iteration cost $J_{0\rightarrow\infty}^{j}(\cdot)$ does not increase with the iteration index $j$.

Proof: Follows from Theorem 2 in Rosolia and Borrelli (2017) $\blacksquare$

Theorem 3

Consider system (1) in closed loop with the LMPC controller (18) and (20) with $N>1$. Let $\mathcal{CS}^{j}$ be the convex safe set at the $j$-th iteration as defined in (10). Let assumption 1 hold and assume that the closed loop system (1) and (20) converges to a steady state trajectory ${\bf{x}}^{\infty}$, for iteration $j\rightarrow\infty$. Then, the steady state input ${\bf{u}}^{\infty}=\lim_{j\to\infty}{\bf{u}}^{j}$ and the related steady state trajectory ${\bf{x}}^{\infty}=\lim_{j\to\infty}{\bf{x}}^{j}$ is a global optimal solution for the infinite horizon optimal control problem (5), i.e., ${\bf{x}}^{\infty}={\bf{x}}^{*}$ and ${\bf{u}}^{\infty}={\bf{u}}^{*}$.

Theorem 4

Consider system (1) in closed loop with the LMPC controller (18) and (20) with $N>1$. Let $\mathcal{CS}^{j}$ be the sampled safe set at the $j$th iteration as defined in (10). Let assumption 1 hold and assume that the closed loop system (1) and (20) converges to a steady state trajectory ${\bf{x}}^{\infty}$, for iteration $j\rightarrow\infty$. Denote $\operatorname{Int}(\mathcal{S})$ as the interior of the set $\mathcal{S}$, and recall the definition of ones-step predecessor $\operatorname{Pre}(\cdot)$ and successor $\operatorname{Succ}(\cdot)$ sets from (Rosolia and Borrelli, 2017, Section II). If $x_{k}^{\infty}\in\operatorname{Int}(\operatorname{Pre}(x_{k+1}^{\infty}))$ and $x_{k+1}^{\infty}\in\operatorname{Int}(\operatorname{Succ}(x_{k}^{\infty}))$ for all $k\geq 0$. Then, the steady state input ${\bf{u}}^{\infty}=\lim_{j\to\infty}{\bf{u}}^{j}$ and the related steady state trajectory ${\bf{x}}^{\infty}=\lim_{j\to\infty}{\bf{x}}^{j}$ is a global optimal solution for the infinite horizon optimal control problem (5), i.e., ${\bf{x}}^{\infty}={\bf{x}}^{*}$ and ${\bf{u}}^{\infty}={\bf{u}}^{*}$.

Proof: Follows from the convexity of (18), (20) and of Problem (5) and Theorem 3 in Rosolia and Borrelli (2017).

$\blacksquare$

4 Example: Constrained LQR controller

In this section, we test the proposed LMPC for linear system on the following infinite horizon linear quadratic regulator with constraints (CLQR)

[TABLE]

In Rosolia and Borrelli (2017) we showed that the LMPC converges to the solution of the infinite horizon control problem (33), whenever we have can compute a feasible trajectory ${\bf{x^{0}}}$. However, the LMPC in Rosolia and Borrelli (2017) is implemented using the sampled Safe Set (8) as a terminal constraint, instead of the proposed convex Safe Set (10). Therefore, also for linear systems, the LMPC presented in Rosolia and Borrelli (2017) involves the solution of a Mixed Integer Programming (MIP) program which is computationally expensive. In the following, we show that the proposed convex formulation of the LMPC for linear systems reduces the computational burden by several order of magnitude and it converges to the solution of the infinite horizon control problem (33).

The LMPC (18), (20) is implemented with the quadratic running cost $h(x_{k},u_{k})=||x_{k}||_{2}^{2}+||u_{k}||_{2}^{2}$, an horizon of $\bar{N}$ steps, and the states and input constraints (33d)-(33e). The LMPC (18) and (20) is reformulated as a Quadratic Programming and it is implemented in YALMIP (Lofberg (2004)) using the solver quadprog. In order to implement the terminal cost (17) we defined the time ${t}_{j}$ at which the iterations is completed,

[TABLE]

with $\epsilon=10^{-8}$.

For $x_{S}=[-3.95,-0.05]^{T}$ and $\bar{N}=4$, the LMPC converges a to steady state solution ${\bf{x}^{\infty}}={\bf{x}^{8}}$, after $8$ iterations, with a error of $\gamma=10^{-10}$:

[TABLE]

Table 1 shows the evolution of the iterations cost. We notice that accordingly with Theorem 2 the cost is non-increasing over the iterations.

Furthermore, the solution of the LMPC for linear system is compared with the exact solution of the CLQR (33), which is computed using the algorithm in Borrelli (2003). Given the optimal solution to the infinite horizon optimal control problem (33),

[TABLE]

we define the approximation error as

[TABLE]

$\sigma_{t}$ quantifies, at each time step $t$, the distance between the optimal trajectory of the CLQR (33) and steady state trajectory at of the LMPC (18) and (20). The maximum approximation error is

[TABLE]

Moreover, the $2$-norm of the normalized difference between the exact optimal cost and the cost associated with the steady state trajectory is

[TABLE]

The LMPC for linear system (18) and (20) has converged to global optimal solution.

We tested the LMPC (18), (20) with different initial conditions $x_{S}$ and horizon length $\bar{N}>1$ to experimentally validate Theorems 1-3. Table 2 shows the maximum approximation error, $\bar{\sigma}$, and $\Delta J$. We underline that for all the tested scenarios, regardless of the horizon length, the proposed LMPC converged to the global optimal solution of the infinite horizon control problem. It is interesting to notice that the LMPC (18), (20) with a longer horizon $\bar{N}$ has more freedom to explore the state space and therefore it converges faster to the steady state trajectory.

Finally, we compare the computational burden associate with the LMPC (18), (20) and with the LMPC in Rosolia and Borrelli (2017). The proposed LMPC (18), (20) applied to Problem (33) converged in $40s$ to a steady state trajectory. On the other hand, the LMPC in Rosolia and Borrelli (2017) applied to Problem (33) took $2hr$ to reach convergence. Therefore, we conclude that the proposed approach significantly reduces the computational burden of the control logic preserving the properties of the LMPC.

5 Conclusions

In this paper, an extension to the learning Model Predictive Control (LMPC) is presented. The controller is designed for linear system and it significantly reduces the computational burden associated with the LMPC. A convex safe set and a terminal cost, learnt from previous iterations, allow to guarantee the recursive feasibility and stability of the closed loop system. Furthermore, the LMPC is guaranteed to improve the performance of the close-loop system over the iterations. We tested the proposed control logic on an infinite horizon linear quadratic regulator with constraints (CLQR) to show that the proposed control logic converges to the optimal solution of the infinite optimal control problem. Finally, we compared the computation time of the proposed strategy with the computational time of the LMPC for nonlinear system, and we showed that the proposed control logic reduces the computational burden by several order of magnitudes.