Formal Synthesis of Control Strategies for Positive Monotone Systems

TL;DR

This paper presents a formal method for synthesizing control strategies for positive monotone systems using signal temporal logic, formulating the problem as a MILP and applying it to traffic management scenarios.

Contribution

It introduces a MILP-based approach for control synthesis for positive monotone systems with STL specifications, establishing conditions where open-loop controls suffice and developing a robust predictive framework.

Findings

Control policies can be synthesized via MILP for STL specifications.

Open-loop control policies are often sufficient for satisfaction.

Application to traffic management demonstrates practical effectiveness.

Abstract

We design controllers from formal specifications for positive discrete-time monotone systems that are subject to bounded disturbances. Such systems are widely used to model the dynamics of transportation and biological networks. The specifications are described using signal temporal logic (STL), which can express a broad range of temporal properties. We formulate the problem as a mixed-integer linear program (MILP) and show that under the assumptions made in this paper, which are not restrictive for traffic applications, the existence of open-loop control policies is sufficient and almost necessary to ensure the satisfaction of STL formulas. We establish a relation between satisfaction of STL formulas in infinite time and set-invariance theories and provide an efficient method to compute robust control invariant sets in high dimensions. We also develop a robust model predictive framework to plan controls optimally while ensuring the satisfaction of the specification. Illustrative examples and a traffic management case study are included.

Tables

Table 1. TABLE I: Parameters of the network in Fig. 5

links	parameters
$1-26,49-53$	${\overline{q}}_l=15,c_l=40$
$27-34$	${\overline{q}}_l=15,c_l=30$
$35-48$	${\overline{q}}_l=40,c_l=60$
Turning ratios	value
$\begin{array}{c}{\beta }_{2:50},{\beta }_{4:53},{\beta }_{8:51},{\beta }_{12:7},{\beta }_{13:28},{\beta }_{15:30},{\beta }_{16:28},\\ {\beta }_{21:32},{\beta }_{24:32},{\beta }_{26:2},{\beta }_{36:31},{\beta }_{36:33},{\beta }_{39:27},{\beta }_{43:29}\end{array}$	0.2
$\begin{array}{c}{\beta }_{5:12},{\beta }_{6:13},{\beta }_{6:18},{\beta }_{10:21},{\beta }_{10:26}\end{array}$	0.3
${\beta }_{1:20},{\beta }_{6:7}$	0.4
$\begin{array}{c}{\beta }_{1:2},{\beta }_{11:12},{\beta }_{14:30},{\beta }_{17:7},{\beta }_{17:18},{\beta }_{19:2},{\beta }_{19:20},\\ {\beta }_{22:34},{\beta }_{23:24},{\beta }_{23:34},{\beta }_{27:14},{\beta }_{27:17},{\beta }_{29:16},{\beta }_{31:22},\\ {\beta }_{31:25},{\beta }_{33:24},{\beta }_{49:3},{\beta }_{49:50},{\beta }_{51:4},{\beta }_{52:5},{\beta }_{52:53},\end{array}$	0.5
$\begin{array}{c}{\beta }_{2:3},{\beta }_{3:4},{\beta }_{4:5},{\beta }_{8:9},{\beta }_{12:13},{\beta }_{13:14},{\beta }_{15:16},{\beta }_{16:17},{\beta }_{20:21},\\ {\beta }_{21:22},{\beta }_{24:25},{\beta }_{25:26},{\beta }_{36:37},{\beta }_{39:40},{\beta }_{43:44},{\beta }_{46:47},\end{array}$	0.8
Capacity ratios	value
$\begin{array}{c}{\alpha }_{19:2},{\alpha }_{26:2},{\alpha }_{17:7},{\alpha }_{12:7},{\alpha }_{13:28},{\alpha }_{16:28}\\ {\alpha }_{14:30},{\alpha }_{15:30},{\alpha }_{21:32},{\alpha }_{24:32},{\alpha }_{22:34},{\alpha }_{23:34}\end{array}$	0.5
Disturbances (arrival rates)
$\begin{array}{c}{w}_1^{\ast }=w_6^{\ast }=4.5,w_{11}^{\ast }=w_{15}^{\ast }=w_{19}^{\ast }=5,w_{23}^{\ast }=6\\ w_{35}^{\ast }=w_{42}^{\ast }=20,w_{49}^{\ast }=w_{52}^{\ast }=2\end{array}$

Table 2. TABLE II: Existence of ϕ italic-ϕ \phi -sequences for the network in Fig. 5

Demand Changes from Table I	$T$	Existence	Comp. Time (s)
-	5	yes	6
-	6	no	4
-	7	no	10
-	8	no	75
-	9	no	11
-	10	yes	36
$w_1^{\ast }=w_6^{\ast }=3,w_{11}^{\ast }=w_{15}^{\ast }=w_{19}^{\ast }=6$	5	yes	5
$w_1^{\ast }=w_6^{\ast }=4,w_{11}^{\ast }=w_{15}^{\ast }=w_{19}^{\ast }=6$	5	no	0.5
$w_1^{\ast }=w_6^{\ast }=1.5,w_{49}^{\ast }=w_{52}^{\ast }=3.5$	5	yes	16
$w_1^{\ast }=w_6^{\ast }=7.5,w_{11}^{\ast }=w_{15}^{\ast }=w_{19}^{\ast }=w_{23}^{\ast }=2$	6	yes	9
$w_1^{\ast }=w_6^{\ast }=9,w_{11}^{\ast }=w_{15}^{\ast }=w_{19}^{\ast }=w_{23}^{\ast }=1$	5	yes	4
$w_1^{\ast }=w_6^{\ast }=10,w_{11}^{\ast }=w_{15}^{\ast }=w_{19}^{\ast }=w_{23}^{\ast }=0$	30	no	3.5
$w_{15}^{\ast }=w_{23}^{\ast }=8,w_{35}^{\ast }=w_{42}^{\ast }=10$	6	yes	23
$w_{15}^{\ast }=w_{23}^{\ast }=0,w_{35}^{\ast }=w_{42}^{\ast }=30$	5	yes	4

Table 3. TABLE III: ϕ italic-ϕ \phi -sequence in the case study

-	Initialization			Repetitive Controls
node	$u_0^{\varphi }$	$u_1^{\varphi }$	$u_2^{\varphi }$	$u_3^{\varphi }$	$u_4^{\varphi }$	$u_5^{\varphi }$	$u_6^{\varphi }$	$u_7^{\varphi }$
$a$	$EW$	$NS$	$NS$	$NS$	$EW$	$EW$	$NS$	$NS$
$b$	$NS$	$EW$	$EW$	$EW$	$NS$	$NS$	$EW$	$EW$
$c$	$EW$	$NS$	$NS$	$EW$	$EW$	$EW$	$NS$	$NS$
$d$	$EW$	$NS$	$EW$	$NS$	$EW$	$EW$	$NS$	$EW$
$e$	$EW$	$EW$	$NS$	$NS$	$NS$	$EW$	$EW$	$NS$
$f$	$NS$	$EW$	$NS$	$EW$	$NS$	$NS$	$EW$	$NS$
$g$	$NS$	$EW$	$NS$	$EW$	$EW$	$NS$	$EW$	$NS$
$h$	$EW$	$NS$	$EW$	$EW$	$EW$	$EW$	$NS$	$EW$
$i$	$EW$	$NS$	$EW$	$EW$	$NS$	$EW$	$NS$	$EW$
$j$	$EW$	$EW$	$NS$	$NS$	$EW$	$EW$	$EW$	$NS$
$k$	$EW$	$EW$	$NS$	$NS$	$NS$	$EW$	$EW$	$NS$
$l$	$NS$	$EW$	$EW$	$NS$	$NS$	$NS$	$EW$	$EW$
$m$	$EW$	$NS$	$NS$	$EW$	$NS$	$EW$	$NS$	$NS$
$n$	$NS$	$NS$	$EW$	$NS$	$EW$	$NS$	$NS$	$EW$
$o$	$0.0$	$0.0$	$0.0$	$12.8$	$0.0$	$0.0$	$0.0$	$0.0$
$p$	$4.0$	$14.0$	$0.0$	$9.5$	$0.0$	$4.0$	$11.5$	$0.0$
$q$	$0.0$	$0.0$	$10.0$	$0.0$	$2.5$	$0.0$	$0.0$	$10.0$
$r$	$5.5$	$0.0$	$4.0$	$14.0$	$11.5$	$5.5$	$0.0$	$4.0$

Equations

$$\phi :=\pi |{\phi }_1\wedge {\phi }_2|{\phi }_1\vee {\phi }_2|{\phi }_1{\mathbf{U}}_I{\phi }_2|{\mathbf{F}}_I\phi |{\mathbf{G}}_I\phi ,$$

$$\begin{array}[]{lll}{\bf{{r}}}[t]\models\pi&\Leftrightarrow&p({r}_{t})\leq c,\\ {\bf{{r}}}[t]\models\varphi_{1}\vee\varphi_{2}&\Leftrightarrow&{\bf{{r}}}[t]\models\varphi_{1}~{}\vee~{}{\bf{{r}}}[t]\models\varphi_{2},\\ {\bf{{r}}}[t]\models\varphi_{1}\wedge\varphi_{2}&\Leftrightarrow&{\bf{{r}}}[t]\models\varphi_{1}~{}\wedge~{}{\bf{{r}}}[t]\models\varphi_{2},\\ {\bf{{r}}}[t]\models\varphi_{1}~{}{\bf U}_{I}~{}\varphi_{2}&\Leftrightarrow&\exists t^{\prime}\in t+I~{}{s.t}~{}{\bf{{r}}}[{t^{\prime}}]\models\varphi_{2}\\ &&\wedge~{}\forall t^{\prime\prime}\in[t,t^{\prime}],{\bf{{r}}}[{t^{\prime\prime}}]\models\varphi_{1},\\ {\bf{{r}}}[t]\models{\bf F}_{I}\varphi&\Leftrightarrow&\exists t^{\prime}\in t+I~{}s.t.~{}{\bf{{r}}}[t^{\prime}]\models\varphi,\\ {\bf{{r}}}[t]\models{\bf G}_{I}\varphi&\Leftrightarrow&\forall t^{\prime}\in t+I~{}s.t.~{}{\bf{{r}}}[t^{\prime}]\models\varphi,\end{array}$$

$$\begin{array}[]{rl}h^{\pi}=&0,\\ h^{\varphi_{1}\wedge\varphi_{2}}=h^{\varphi_{1}\vee\varphi_{2}}=&\max(h^{\varphi_{1}},h^{\varphi_{2}}),\\ h^{{\bf F}_{[t_{1},t_{2}]}\varphi}=h^{{\bf G}_{[t_{1},t_{2}]}\varphi}=&t_{2}+h^{\varphi},\\ h^{\varphi_{1}{\bf U}_{[t_{1},t_{2}]}\varphi_{2}}=&t_{2}+\max(h^{\varphi_{1}},h^{\varphi_{2}}).\\ \end{array}$$

$$\begin{array}[]{lll}\rho({\bf{{r}}},\pi,t)&=&c-p({r_{t}}),\\ \rho({\bf{{r}}},\varphi_{1}\vee\varphi_{2},t)&=&\max(\rho({\bf{{r}}},\varphi_{1},t),\rho({\bf{{r}}},\varphi_{2},t)),\\ \rho({\bf{{r}}},\varphi_{1}\wedge\varphi_{2},t)&=&\min(\rho({\bf{{r}}},\varphi_{1},t),\rho({\bf{{r}}},\varphi_{2},t)),\\ \rho({\bf{{r}}},\varphi_{1}~{}{\bf U}_{I}~{}\varphi_{2},t)&=&\underset{t^{\prime}\in t+I}{\max}\big{(}\min(\rho({\bf{{r}}},\varphi_{2},t^{\prime}),\\ &&~{}~{}~{}~{}~{}~{}~{}\underset{t^{\prime\prime}\in[t,t^{\prime}]}{\min}\rho({\bf{{r}}},\varphi_{1},t^{\prime\prime}))\big{)},\\ \rho({\bf{{r}}},{\bf F}_{I}~{}\varphi,t)&=&\underset{t^{\prime}\in t+I}{\max}~{}\rho({\bf{{r}}},\varphi,t^{\prime}),\\ \rho({\bf{{r}}},{\bf G}_{I}~{}\varphi,t)&=&\underset{t^{\prime}\in t+I}{\min}~{}\rho({\bf{{r}}},\varphi,t^{\prime}).\\ \end{array}$$

$$x_{t+1}=f(x_t,u_t,w_t),$$

$$\forall x\in \mathcal{X},\forall u\in \mathcal{U},f(x,u,w)⪯f(x,u,w^{\ast }),\forall w\in \mathcal{W}.$$

$$\pi =\left(a_{\pi }^{T}x\le b_{\pi }\right),$$

$$u_t={\mu }_t(x_0,x_1,\cdots ,x_t).$$

$${\bf{{x}}}={\bf{{x}}}(x_{0},\mu,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}{{\bf{{w}}}}}):=x_{0}x_{1}x_{2}\cdots,$$

$$\mathbf{x}(x_0,\mu ,\mathbf{w})[0]\models \phi ,\forall \mathbf{w}\in {\mathcal{W}}^{\omega },\forall x_0\in {\mathcal{X}}_0.$$

$${\phi }_b\wedge {\mathbf{G}}_{[\Delta ,\infty ]}{\phi }_g,$$

$$\mathbf{x}(x_0,\mu ,\mathbf{w})[0]\models {\phi }_b\wedge {\mathbf{G}}_{[\Delta ,\infty ]}{\phi }_g,\forall \mathbf{w}\in {\mathcal{W}}^{\omega },\forall x_0\in {\mathcal{X}}_0.$$

$$x_t^H(x_t,u_t^H,w_t^H):=x_{1|t}{x}_{2|t}\cdots x_{H|t},$$

$$\begin{array}[]{cl}\displaystyle\operatorname*{arg\,min}_{u_{t}^{H}}&\underset{{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}{w^{H}_{t}}}}{\max}~{}J\left(x_{t}^{H}(x_{t},u_{t}^{H},w_{t}^{H}),u_{t}^{H}\right),\\ \text{s.t.}&{\bf{{x}}}(x_{0},\mu,{\bf{{w}}})[0]\models{\bf{{G}}}_{[0,\infty]}\varphi,\forall{\bf{{w}}}\in\mathcal{W}^{\omega},\\ &x_{k+1}=f(x_{k},u_{k},w_{k}),\forall k\in\mathbb{N}.\\ \end{array}$$

$$\mathcal{X}_{0}^{\max}:=\left\{x_{0}\in\mathcal{X}{\Big{|}}\exists\mu~{}\text{s.t.}~{}{\bf{{x}}}(x_{0},\mu,{\bf{{w}}})\models\varphi,\forall{\bf{{w}}}\in\mathcal{W}^{\omega}\right\}.$$

$$u_0^{ol,x_0}{u}_1^{ol,x_0}\cdots u_{h^{\phi }-1}^{ol,x_0}$$

$${\mu }_t^{ol}(x_0)=u_t^{ol,x_0},t=0,1,\cdots ,h^{\phi }-1,$$

$$a_{\pi }^{T}x-M(1-z^{\pi })+\rho \le b_{\pi },$$

$$a_{\pi }^{T}x+M{z}^{\pi }+\rho \ge b_{\pi }.$$

$$z=\underset{i=1}{\overset{n_z}{\bigwedge }}{z}_i\Rightarrow z\le z_i,i=1,\cdots ,n_z,$$

$$z=\underset{i=1}{\overset{n_z}{\bigvee }}{z}_i\Rightarrow z\le \sum _{i=1}^{n_z}{z}_i,$$

$$\begin{array}[]{rl}\varphi=\bigwedge_{i=1}^{n_{\varphi}}\varphi_{i}\Rightarrow&z^{\varphi}_{k}=\bigwedge_{i=1}^{n_{\varphi}}z^{\varphi_{i}}_{k},\\ \varphi=\bigvee_{i=1}^{n_{\varphi}}\varphi_{i}\Rightarrow&z^{\varphi}_{k}=\bigvee_{i=1}^{n_{\varphi}}z^{\varphi_{i}}_{k},\\ \varphi={\bf{{G}}}_{I}\psi\Rightarrow&z^{\varphi}_{k}=\bigwedge_{k^{\prime}\in I}z^{\psi}_{k^{\prime}},\\ \varphi={\bf{{F}}}_{I}\psi\Rightarrow&z^{\varphi}_{k}=\bigvee_{k^{\prime}\in I}z^{\psi}_{k^{\prime}},\\ \varphi=\psi_{1}{\bf U}_{I}\psi_{2}\Rightarrow&z^{\varphi}_{k}=\bigvee_{k^{\prime}\in I}\left(z^{\psi_{2}}_{k^{\prime}}\wedge\bigwedge_{k^{\prime\prime}\in[k,k^{\prime}]}z^{\psi_{1}}_{k^{\prime\prime}}\right).\end{array}$$

$$z_0^{\phi }=1,\rho \ge 0.$$

$$\begin{array}[]{rl}\rho^{\max}=\text{argmax}&\rho,\\ \text{s.t.}&a^{T}_{\pi_{i}}x+\rho\leq b_{\pi_{i}},i=1,\cdots,n_{z},\end{array}$$

$$\begin{array}[]{rl}\rho^{\max}=\text{argmax}&\rho,\\ \text{s.t.}&\exists i\in\{1,\cdots,n_{z}\},a^{T}_{\pi_{i}}x+\rho\leq b_{\pi_{i}},\end{array}$$

$$x_{t+1}=A{x}_t+B_u{u}_t+B_w{w}_t+D_{\delta }{\delta }_t+D_r{r}_t,$$

$$E_{\delta }{\delta }_t+E_r{r}_t⪯E_x{x}_t+E_u{u}_t+E_w{w}_t+e,$$

$$\left\{\begin{array}[]{ll}x_{0}^{ol,x_{0}}=x_{0},&\text{Initial condition;}\\ x_{k+1}^{ol,x_{0}}=f^{*}(x_{k}^{ol,x_{0}},u_{k}^{ol,x_{0}}),&\text{System constraints;}\\ z^{\pi}_{k}=(a_{\pi}^{T}x_{k}^{ol,x_{0}}\leq b_{\pi}),&\text{Predicates; }\\ z^{\varphi}_{0}=1,\rho\geq 0,&\text{STL satisfaction.}\\ \end{array}\right.$$

$$x^{+}=e^{A_u\tau }x+A_u^{-1}(I-e^{-A_u\tau })w,$$

$$A_{1}=\left(\begin{array}[]{cc}1&1\\ 1&-5\end{array}\right),A_{2}=\left(\begin{array}[]{cc}-8&1\\ 1&2\end{array}\right).$$

$$\phi =\underset{T=0}{\overset{10}{\bigvee }}\left({\mathbf{F}}_{[0,T]}{p}_1\wedge {\mathbf{F}}_{\{T\}}{p}_2\right),$$

$$\mathcal{L}^{\varphi}:=\left\{x_{0}x_{1}\cdots x_{h^{\varphi}}\in\mathcal{X}^{h^{\varphi}}\big{|}x_{0}x_{1}\cdots x_{h^{\varphi}}\models\varphi\right\}.$$

Full text

I Introduction

In recent years, there has been a growing interest in using formal methods for specification, verification, and synthesis in control theory. Temporal logics [1] provide a rich, expressive framework for describing a broad range of properties such as safety, liveness, and reactivity. In formal synthesis, the goal is to control a dynamical system from such a specification. For example, in an urban traffic network, a synthesis problem can be to generate traffic light control policies that ensure gridlock avoidance and fast enough traffic through a certain road, for all times.

Control synthesis for linear and piecewise affine systems from linear temporal logic (LTL) specifications was studied in [2, 3, 4]. The automata-based approach used in these works requires constructing finite abstractions that (bi)simulate the original system. Approximate finite bisimulation quotients for nonlinear systems were investigated in [5, 6]. The main limitations of finite abstraction approaches are the large computational burden of discretization in high dimensions and conservativeness when exact bisimulations are impossible or difficult to construct. As an alternative approach, LTL optimization-based control of mixed-logical dynamical (MLD) systems [7] using mixed-integer programs was introduced in [8, 9], and was recently extended to model predictive control (MPC) from signal temporal logic (STL) specifications in [10, 11, 12]. However, these approaches are unable to guarantee infinite-time safety and the results are fragile in the presence of non-deterministic disturbances.

In some applications, the structural properties of the system and the specification can be exploited to consider alternative approaches to formal control synthesis. We are interested in systems in which the evolution of the state exhibits a type of order preserving law known as monotonicity, which is common in models of transportation, biological, and economic systems [13, 14, 15, 16]. Such systems are also positive in the sense that the state components are always non-negative. Control of positive systems have been widely studied in the literature [17, 18, 19]. Positive linear systems are always monotone [20].

In this paper, we study optimal STL control of discrete-time positive monotone systems (i.e., systems with state partial order on the positive orthant) with bounded disturbances. STL allows designating time intervals for temporal operators, which makes it suitable for describing requirements with deadlines. Moreover, STL is equipped with quantitative semantics, which provides a measure to quantify how strongly the specification is satisfied/violated. The quantitative semantics of STL can also be used as cost for maximization in an optimal control setting. The STL specifications in this paper are restricted to a particular form that favors smaller values for the state components. We assume that there exists a maximal disturbance element that characterizes a type of upper-bound for the evolution of the system. These assumptions are specifically motivated by the dynamics of traffic networks, where the disturbances represent the volume of exogenous vehicles entering the network and the maximal disturbance characterizes the rush hour exogenous flow. Our optimal control study is focused on STL formulae with infinite-time safety/persistence properties, which is relevant to optimal and correct traffic control in the sense that the vehicular flow is always free of congestion while the associated delay is minimized.

The key contributions of this paper are as follows. First, for finite-time semantics, we prove that the existence of open-loop control policies is necessary and sufficient for maintaining STL correctness. For the correctness of infinite-time semantics, we show that the existence of open-loop control sequences is sufficient and almost necessary, in a sense that is made clear in the paper. Implementing open-loop control policies is very simple since online state measurements are not required, which can prove useful in applications where the state is difficult to access. We use a robust MPC approach to optimal control. The main contribution of our MPC framework is guaranteed recursive feasibility, a property that was not established in prior STL MPC works [10, 11, 12]. We show via a case study that our method is applicable to systems with relatively high dimensions.

This remainder of the paper is organized as follows. We introduce the necessary notation and background on STL in Sec. II. The problems are formulated in Sec. III. The technical details for control synthesis from finite and infinite-time specifications are given in Sec. IV and Sec. V, respectively. The robust MPC framework is explained in Sec. VI. Finally, we introduce a traffic network model and explain its monotonicity properties in Sec. VII, where a case study is also presented.

Related Work

This paper is an extension of the conference version [21], where we studied safety control of positive monotone systems. Here, we significantly enrich the range of specifications to STL, provide complete proofs, and include optimal control.

Monotone dynamical systems have been extensively investigated in the mathematics literature [22, 23, 24]. Early studies mainly focused on stability properties and characterization of limit sets for autonomous, deterministic continuous-time systems [25, 22]. The results do not generally hold for discrete-time systems, as discussed in [23]. In particular, attractive periodic orbits are proven to be non-existent for continuous-time autonomous systems [25], but may exist for discrete-time autonomous systems. Here we present a similar result for controlled systems, where we show that a type of attractive periodic orbit exists for certain control policies.

Angeli and Sontag [26] extended the notion of monotonicity to deterministic continuous-time control systems and provided results on interconnections of these systems. However, they assumed monotonicity with respect to both state and controls. We do not require monotonicity with respect to controls, which enables us to consider a broader class of systems. In particular, we do not require controls to belong to a partially ordered set.

Switching policies for exponential stabilization of switched positive linear systems were studied in [27, 28]. Stabilization is closely related to set-invariance, which is thoroughly studied in this paper. Apart from richer specifications, we can handle more complex systems. We consider hybrid systems in which the mode is either determined directly by the control input or indirectly by the state (e.g., signalized traffic networks).

Recently, there has been some interest on formal verification and synthesis for monotone systems. Safety control of cooperative systems was investigated in [29, 30, 31]. However, these work, like [26], assumed monotonicity with respect to the control inputs as well. Computational benefits gained from monotonicity for reachability analysis of hybrid systems were highlighted in [32]. More recently, the authors in [33] provided an efficient method to compute finite abstractions for mixed-monotone systems (a more general class than monotone systems). The authors in [34] exploited monotonicity to compute finite-state abstractions that are used for compositional LTL control. While the approaches in [33, 34] can consider systems and specifications beyond the assumptions in this paper, they still require state-space discretization, which is a severe limitation in high dimensions. Moreover, they are conservative since the finite abstractions are often not bisimilar with the original system - whereas our approach provides a notion of (almost) completeness. Finally, as opposed to the all mentioned works, our framework is amenable to optimal temporal logic control.

II Preliminaries

II-A Notation

For two integers $a,b$, we use $\operatorname*{rem}(a,b)$ to denote the remainder of division of $a$ by $b$. Given a set $\mathcal{S}$ and a positive integer $K$, we use the shorthand notation $\mathcal{S}^{K}$ for $\prod_{i=1}^{K}\mathcal{S}$. A signal is defined as an infinite sequence ${\bf{{s}}}=s_{0}s_{1}\cdots$, where $s_{k}\in\mathcal{S}$, $k\in\mathbb{N}$. Given $s_{1},s_{2},\cdots,s_{K}\in\mathcal{S}$, the repetitive infinite-sequence $s_{1}s_{2}\cdots s_{K}s_{1}s_{2}\cdots s_{K}\cdots$ is denoted by $(s_{1}s_{2}\cdots s_{K})^{\omega}$. The set of all signals that can be generated from $\mathcal{S}$ is denoted by $\mathcal{S}^{\omega}$. We use ${\bf{{s}}}[k]={s}_{k}{s}_{k+1}\cdots$ and ${\bf{{s}}}[{k_{1}:k_{2}}]={s}_{k_{1}}{s}_{k_{1}+1}\cdots{s}_{k_{2}}$, $k_{1}<k_{2}$, to denote specific portions of ${\bf{{s}}}$. A real signal is ${\bf{{{\bf{{r}}}}}}={r}_{0}{r}_{1}{r}_{2}\cdots$, where ${r}_{k}\in\mathbb{R}^{n},\forall k\in\mathbb{N}$. A vector of all ones in $\mathbb{R}^{n}$ is denoted by $1_{n}$. We use the notation ${\bf{{1}}}_{n}[0:K]:={1_{n}\cdots 1_{n}}$, where $1_{n}$ is repeated $K+1$ times. The positive closed orthant of the $n$-dimensional Euclidian space is denoted by $\mathbb{R}^{n}_{+}:=\left\{x\in\mathbb{R}^{n}|x_{[i]}\geq 0,i=1,\cdots,n\right\}$, where $x=(x_{[1]},x_{[2]},\cdots,x_{[n]})^{T}$. For $a,b\in\mathbb{R}^{n}$, the non-strict partial order relation $\preceq$ is defined as: $a\preceq b\Leftrightarrow b-a\in\mathbb{R}_{+}^{n}.$

Definition 1* ([35]).*

A set $\mathcal{X}\subset\mathbb{R}^{n}_{+}$ is a lower-set if $\forall x\in\mathcal{X},L(x)\subseteq\mathcal{X}$, where L(x):=\left\{y\in\mathbb{R}_{+}^{n}\big{|}y\preceq x\right\}.

It is straightforward to verify that if $\mathcal{X}_{1},\mathcal{X}_{2}$ are lower-sets, then $\mathcal{X}_{1}\cup\mathcal{X}_{2}$ and $\mathcal{X}_{1}\cap\mathcal{X}_{2}$ are also lower-sets. We extend the usage of notation $\preceq$ to equal-length real signals. For two real signals ${\bf{{r}}},{\bf{{r}}}$, we denote ${\bf{{r}}}^{\prime}[t_{1}^{\prime}:t_{2}^{\prime}]\preceq{\bf{{r}}}[t_{1}:t_{2}]$, $t_{2}-t_{1}=t^{\prime}_{2}-t^{\prime}_{1}$, if $r^{\prime}_{t^{\prime}_{1}+k}\preceq r_{t_{1}+k},k=0,1,\cdots,t_{2}-t_{1}$. Moreover, if ${\bf{{r}}},{\bf{{r}}}^{\prime}\in({\mathbb{R}_{+}^{n}})^{\omega}$, we are also allowed to write ${\bf{{r}}}^{\prime}[t_{1}^{\prime}:t_{2}^{\prime}]\in L({\bf{{r}}}[t_{1}:t_{2}])$.

II-B Signal Temporal Logic (STL)

In this paper, STL [36] formulas are defined over discrete-time real signals. The syntax of negation-free STL is:

[TABLE]

where $\pi=(p({r})\leq c)$ is a predicate on ${r}\in\mathbb{R}^{n}$, $p:\mathbb{R}^{n}\rightarrow\mathbb{R}$, $c\in\mathbb{R}$; $\wedge$ and $\vee$ are Boolean connectives for conjunction and disjunction, respectively; ${\bf U}_{I}$, ${\bf{{F}}}_{I}$, ${\bf{{G}}}_{I}$ are the timed until, eventually and always operators, respectively, and $I=[t_{1},t_{2}]$ is a time interval, $t_{1},t_{2}\in\mathbb{N}\cup\{\infty\},t_{2}\geq t_{1}$. When $t_{1}=t_{2}$, we use the shorthand notation $\left\{t_{1}\right\}:=[t_{1},t_{1}]$. Exclusion of negation does not restrict expressivity of temporal properties. It can be easily shown that any temporal logic formula can be brought into negation normal form (where all negation operators apply to the predicates) [37, 12]. We deliberately omit negation from STL syntax for laying out properties that are later exploited in the paper. For simplicity, in the rest of the paper, we will refer to negation-free STL simply as STL. The semantics of STL is inductively defined as:

[TABLE]

where $\models$ is read as satisfies. The language of $\varphi$ is the set of all signals such that ${\bf{{r}}}[0]\models\varphi$. The horizon of an STL formula $\varphi$, denoted by $h^{\varphi}$, is defined as the time required to decide the satisfaction of $\varphi$, which is recursively computed as [38]:

[TABLE]

Definition 2*.*

An STL formula $\varphi$ is bounded if $h^{\varphi}<\infty$.

Definition 3* ([39]).*

A safety STL formula is an STL formula in which all “until” and “eventually” intervals are bounded.

The satisfaction of $\varphi$ by ${\bf{{r}}}[t]$ is decided only by ${\bf{{r}}}[t:t+h^{\varphi}]$ and the rest of the signal values are irrelevant. Therefore, instead of ${\bf{{r}}}[t]\models\varphi$, we occasionally write ${\bf{{r}}}[t:t+h^{\varphi}]\models\varphi$ with the same meaning. The STL robustness score $\rho({\bf{{r}}},\varphi,t)\in\mathbb{R}$ is a measure indicating how strongly $\varphi$ is satisfied by ${\bf{{r}}}[t]$, which is recursively computed as [36]:

[TABLE]

Positive (respectively, negative) robustness indicates satisfaction (respectively, violation) of the formula.

Example 1*.*

Consider signal ${\bf{{r}}}\in\mathbb{R}^{\omega}$, where ${r}_{k}=k,k\in\mathbb{N}$, and $\pi=({r}^{2}\leq 10)$. We have $\rho({\bf{{r}}},{\bf{{G}}}_{[0,3]}\pi,0)=\min(10-0,10-1,10-4,10-9)=1$ (satisfaction) and $\rho({\bf{{r}}},{\bf{{F}}}_{[4,6]}\pi,0)=\max(10-16,10-25,10-36)=-6$ (violation).

Remark 1*.*

There are minor differences between the original STL introduced in [36] and the one used in this paper. In [36], STL was developed as an extension of metric interval temporal logic (MITL) [39] for real-valued continuous-time signals. Here, without any loss of generality, we apply STL to discrete-time signals. Our STL is based on metric temporal logic (MTL) (similar to [38]). Thus, we allow the intervals of temporal operators to be singletons (punctual) or unbounded. It is worth to note that any STL formula in this paper can be translated into an LTL formula by appropriately replacing the time intervals of temporal operators with LTL “next” operator. However, the LTL representation of STL formulas can be very inefficient. We prefer STL for convenience of specifying requirements for systems with real-valued states. We also exploit the STL quantitative semantics.

III Problem Statement and Approach

We consider discrete-time systems of the following form:

[TABLE]

where $x_{t}\in\mathcal{X}$ is the state, $\mathcal{X}\subset\mathbb{R}_{+}^{n}$, $u_{t}\in\mathcal{U}$ is the control input, $\mathcal{U}=\mathbb{R}^{m_{r}}\times\{0,1\}^{m_{b}}$, and $w_{t}\in\mathcal{W}$ is the disturbance (adversarial input) at time $t$, $t\in\mathbb{N}$, $\mathcal{W}=\mathbb{R}^{q_{r}}\times\{0,1\}^{q_{b}}$. The sets $\mathcal{U}$ and $\mathcal{W}$ may include real and binary values. For instance, the set of controls in the traffic model developed in Sec. VII includes binary values for decisions on traffic lights and real values for ramp meters. These types of systems are positive as all state components are non-negative. We also assume that $\mathcal{X}$ is bounded.

Definition 4*.*

System (5) is monotone (with partial order on $\mathbb{R}^{n}_{+}$) if for all $x,x^{\prime}\in\mathcal{X}$, $x^{\prime}\preceq x$, we have $f(x^{\prime},u,w)\preceq f(x,u,w),\forall u\in\mathcal{U},\forall w\in\mathcal{W}$.

The systems considered in this paper are positive and monotone with partial order on $\mathbb{R}^{n}_{+}$. For the remainder of the paper, we simply refer to systems in Definition 4 as monotone 111The term cooperative in dynamics systems theory is used specifically to refer to systems that are monotone with partial order defined on the positive orthant. We avoid using this term here as it might generate confusion with the similar terminology used for multi-agent control systems.. Although the results of this paper are valid for any general $f:\mathcal{X}\times\mathcal{U}\times\mathcal{W}\rightarrow\mathcal{X}$, we focus on systems that can be written in the form of mixed-logical dynamical (MLD) systems [7], which are defined in Sec. IV. It is well known that a wide range of systems involving discontinuities (hybrid systems), such as piecewise affine systems, can be transformed into MLDs [40].

Assumption 1*.*

There exist $w^{*}\in\mathcal{W}$ such that

[TABLE]

We denote $f(x,u,w^{*})$ by $f^{*}(x,u)$ and refer to $f^{*}$ as the maximal system. As it will be further explained in this paper, the behavior of monotone system (5) is mainly characterized by its maximal $f^{*}$. Assumption 1 is restrictive but holds for many compartmental systems where the disturbances are additive and the components are independent. Therefore, the maximal system corresponds to the situation that every component takes its most extreme value. We also note that if Assumption 1 is removed, overestimating $f$ by some $f^{*}$ such that $f(x,u,w)\preceq f^{*}(x,u),\forall w\in\mathcal{W}$, is always possible for a bounded $f$. By overestimating $f$ the control synthesis methods of this paper remain correct, but become conservative.

We describe the desired system behavior using specifications written as STL formulas over a finite set of predicates. We assume that each predicate $\pi$ is in the following form:

[TABLE]

where $a_{\pi}\in\mathbb{R}^{n}_{+}$, $b_{\pi}\in\mathbb{R}_{+}$. It is straightforward to verify that the closed half-space defined by (7) is a lower-set in $\mathbb{R}^{n}_{+}$. By restricting the predicates into the form (7), we ensure that a predicate remains true if the values of state components are decreased (Note that this is true for any lower set. We require linearity in order to decrease the computational complexity.). This restriction is motivated by monotonicity. For example, in a traffic network, the state is the vector representation of vehicular densities in different segments of the network. The satisfaction of a “sensible” traffic specification has to be preserved if the vehicular densities are not increased all over the network. Otherwise, the specification encourages large densities and congestion.

Definition 5*.*

A control policy $\mu:=\bigcup_{t\in\mathbb{N}}\mu_{t}$ is a set of functions $\mu_{t}:\mathcal{X}^{t+1}\rightarrow\mathcal{U}$, where

[TABLE]

An open-loop control policy takes the simpler form $u_{t}=\mu_{t}(x_{0})$, i.e., the decision on the sequence of control inputs is made using only the initial state $x_{0}$. On the other hand, in a (history dependent) feedback control policy, $u_{t}=\mu_{t}(x_{0},x_{1},\cdots,x_{t})$, the controller implementation requires real-time access to the state and its history.

An infinite sequence of admissible disturbances is ${\bf{{w}}}=w_{0}w_{1}\cdots$, where $w_{k}\in\mathcal{W}$, $k\in\mathbb{N}$. Following the notation introduced in Sec. II-A, the set of all infinite-length sequences of admissible disturbances is denoted by $\mathcal{W}^{\omega}$. Given an initial condition $x_{0}$, a control policy $\mu$ and ${\bf{{w}}}\in\mathcal{W}^{\omega}$, the run of the system is defined as the following signal:

[TABLE]

where $x_{t+1}=f(x_{t},u_{t},w_{t}),\forall t\in\mathbb{N}$. Now we formulate the problems studied in this paper. In all problems, we assume a monotone system (5) is given, Assumption 1 holds, and all the predicates are in the form of (7).

Problem 1** (Bounded STL Control).**

Given a bounded STL formula $\varphi$, find a set of initial conditions $\mathcal{X}_{0}\subset\mathcal{X}$ and a control policy $\mu$ such that

[TABLE]

As mentioned in the previous section, the satisfaction of $\varphi$ solely depends on ${\bf{{x}}}[0:h^{\varphi}]$, where $h^{\varphi}$ is obtained from (3). The horizon $h^{\varphi}$ can be viewed as the time when the specification ends. In many engineering applications, the system is required to uphold certain behaviors for all times. Therefore, guaranteeing infinite-time safety properties is important. We formulate bounded-global STL formulas in the form of

[TABLE]

where $\varphi_{b},\varphi_{g}$ are bounded STL formulas, ${\bf G}_{[\Delta,\infty]}$ stands for unbounded temporal “always”- as defined in Sec. II-B, and $\Delta\geq h^{\varphi_{b}}$ is a positive integer. Formula (8) states that first, $\varphi_{b}$ is satisfied by the signal from time 0 to $\Delta$, and, afterwards, $\varphi_{g}$ holds for all times.

Problem 2** (Bounded-global STL Control).**

Given bounded STL formulas $\varphi_{b},\varphi_{g}$, $\Delta\in[h^{\varphi_{b}},\infty)$, find a set of initial conditions $\mathcal{X}_{0}\subset\mathcal{X}$ and a control policy $\mu$ such that

[TABLE]

As a special case, we allow $\varphi_{b}$ to be logical truth so Problem 2 reduces to global STL control problem of satisfying ${\bf G}_{[\Delta,\infty]}\varphi_{g}$. Note that if $\varphi_{g}$ is replaced by logical truth, Problem 2 reduces to Problem 1. We have distinguished Problem 1 and Problem 2 as we use different approaches to solve them.

It can be shown that (see Appendix) a large subset of safety STL formulas - as in Definition 3 - can be written as $\bigvee_{i=1}^{n_{\phi}}\phi_{i},$ where each $\phi_{i},i=1,\cdots,{n_{\phi}}$, is a bounded-global formula. Therefore, the framework for solutions to Problem 2 can also be used for safety STL control as it leads to $n_{\phi}$ instances of Problem 2, where a solution to any of the instances is also a solution to the original safety STL control problem. The drawback to this approach is that $n_{\phi}$ can be very large.

Remark 2*.*

We avoid separate problem formulations for STL formulas containing unbounded “eventually” or “until” operators as their unbounded intervals can be safely under-approximated by bounded intervals. However, bounded under-approximation is not sound for the unbounded “always” operator. A safety formula can be satisfied (respectively, violated) with infinite-length (respectively, finite-length) signals [39].

In the presence of disturbances, feedback controllers obviously outperform open-loop controllers. We show that the existence of open-loop control policies for guaranteeing the STL correctness of monotone systems in Problem 1 (respectively, Problem 2) is sufficient and (respectively, almost) necessary. The online knowledge of state is not necessary for STL correctness. But it can be exploited for planning controls optimally. While our framework can accommodate optimal control versions of Problem 1 and Problem 2, the focus of this paper is on robust optimal control problem for global STL formulas - of form ${\bf{{G}}}_{[0,\infty)}\varphi$, where $\varphi$ is a bounded formula. These type of problems are of practical interest for optimal traffic management (as discussed in Sec. VII).

We use a model predictive control (MPC) approach, which is a popular, powerful approach to optimal control of constrained systems. Given a planning horizon of length $H$ 222The MPC horizon $H$ should not be confused with the STL horizon $h^{\varphi}$. , a sequence of control actions starting from time $t$ is denoted by $u_{t}^{H}:=u_{0|t}u_{1|t}\cdots u_{H-1|t}.$ Given $u^{H}_{t}$ and $x_{t}$, we denote the predicted $H$-step system response by

[TABLE]

where $x_{k+1|t}=f(x_{k|t},u_{k|t},w_{k|t}),k=0,1,\cdots,H-1,$ $x_{0|t}=x_{t}$ and $w_{t}^{H}:=w_{0|t}w_{1|t}\cdots w_{H-1|t}$. At each time, $u^{H}_{t}$ is found such that it optimizes a cost function $J\left(x_{t}^{H},u^{H}_{t}\right)$, $J:\mathcal{X}^{H}\times\mathcal{U}^{H}\rightarrow\mathbb{R}$, subject to system constraints. When $u_{t}^{H}$ is computed, only the first control action $u_{0|t}$ is applied to the system and given the next state, the optimization problem is resolved for $u_{t+1}^{H}$. Thus, the implementation is closed-loop.

Problem 3** (Robust STL MPC).**

Given a bounded STL formula $\varphi$, an initial condition $x_{0}$, a planning horizon $H$ and a cost function $J\left(x_{t}^{H},u^{H}_{t}\right)$, find a control policy such that $u_{t}=\mu(x_{0},\cdots,x_{t})=u^{\text{opt}}_{0|t}$, where $u_{t}^{H,\text{opt}}:=u^{\text{opt}}_{0|t}\cdots u^{\text{opt}}_{H-1|t}$, and $u^{H,\text{opt}}$ is the following minimizer:

[TABLE]

The primary challenge of robust STL MPC is guaranteeing the satisfaction of the global STL formula while the controls are planned in a receding horizon manner (see the constraints in (10)). Our approach takes the advantage of the results from Problem 2 to design appropriate terminal sets for the MPC algorithm such that the generated runs are guaranteed to satisfy the global STL specification while the online control decisions are computed (sub)optimally. Due to the temporal logic constraints, our MPC setup differs from the conventional one. The details are explained in Sec. VI.

For computational purposes, we assume that $J$ is a piecewise affine function of the state and controls. Moreover, the cost functions in our applications are non-decreasing with respect to the state in the sense that x^{\prime}_{k|t}\preceq x_{k|t},k=1,2,\cdots,H\Rightarrow J({\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}{x_{t}^{\prime H}}},u^{H}_{t})\preceq J(x_{t}^{H},u^{H}_{t}),\forall u^{H}_{t}\in\mathcal{U}^{H}. As it will become clear later in the paper, we will exploit this property to simplify the worst-case optimization problem in (10) to an optimization problem for the maximal system.

As mentioned earlier, a natural objective is maximizing STL robustness score. It follows from the linearity of the predicates in (7) and $\max$ and $\min$ operators in (4) that STL robustness score is a piecewise affine function of finite-length signals. We can also consider optimizing a weighted combination of STL robustness score and a given cost function. We use this cost formulation for traffic application in Sec. VII.

IV Finite Horizon Semantics

In this section, we explain the solution to Problem 1. First, we exploit monotonicity to characterize the properties of the solutions. Next, we explain how to synthesize controls using a mixed integer linear programming (MILP) solver.

Lemma 1*.*

Consider runs ${\bf{{x}}}$ and ${\bf{{x}}}^{\prime}$ and an STL formula $\varphi$. If for some $t,t^{\prime}$, we have ${\bf{{x}}}^{\prime}[t^{\prime}:t^{\prime}+h^{\varphi}]\preceq{\bf{{x}}}[t:t+h^{\varphi}]$, then ${\bf{{x}}}[t]\models\varphi$ implies ${\bf{{x}}}^{\prime}[{t^{\prime}}]\models\varphi$.

Proof:

Since all predicates denote lower-sets in the form of (7), we have $x^{\prime}_{t^{\prime}}\preceq x_{t}\Rightarrow a_{\pi}^{T}x^{\prime}_{t^{\prime}}\leq a_{\pi}^{T}x_{t}$, ${\bf{{x}}}[t]\models\pi\Rightarrow{\bf{{x}}}^{\prime}[t]\models\pi$. Thus, all predicates that were true by the valuations in ${\bf{{x}}}$ remain true for ${\bf{{x}}}^{\prime}$. The negation-free semantics in (2) implies that without falsifying any predicate, a formula can not be falsified. Therefore, ${\bf{{x}}}[t]\models\varphi$ implies ${\bf{{x}}}^{\prime}[{t^{\prime}}]\models\varphi$ ∎

The largest set of admissible initial conditions is defined as:

[TABLE]

The set $\mathcal{X}_{0}^{\max}$ is a union of polyhedra. Finding the half-space representation of all polyhedral sets in $\mathcal{X}_{0}^{\max}$ may not be possible for high dimensions. Therefore, we find a half-space representation for a subset of $\mathcal{X}_{0}^{\max}$. The following result states how to check whether $x_{0}\in\mathcal{X}_{0}^{\max}$.

Theorem 1*.*

We have $x_{0}\in\mathcal{X}_{0}^{\max}$ if and only if there exists an open-loop control sequence

[TABLE]

such that ${\bf{{x}}}^{ol,x_{0}}[{0:h^{\varphi}}]\models\varphi$, where ${\bf{{x}}}^{ol,x_{0}}[{0:h^{\varphi}}]=x_{0}^{ol,x_{0}}x_{1}^{ol}\cdots x_{h^{\varphi}}^{ol,x_{0}}$, and $x_{k+1}^{ol,x_{0}}=f^{*}(x_{k}^{ol,x_{0}},u_{k}^{ol,x_{0}}),k=0,\cdots,h^{\varphi}-1,x_{0}^{ol,x_{0}}=x_{0}$.

Proof:

(Necessity) Satisfaction of $\varphi$ with ${\bf{{w}}}\in\mathcal{W}^{\omega}$ requires at least one satisfying run for the maximal system, hence a corresponding control sequence exists. Denote it by $u_{0}^{ol,x_{0}}u_{1}^{ol,x_{0}}\cdots,u_{h^{\varphi}-1}^{ol,x_{0}}$. (Sufficiency) Consider any run generated by the original system $x_{k+1}=f(x_{k},u_{k}^{ol,x_{0}},w_{k})$. We prove that $x_{k}\preceq x_{k}^{ol,x_{0}}~{},k=0,1,\cdots,h^{\varphi}$, by induction over $k$. The base case $x_{0}\preceq x_{0}^{ol,x_{0}}$ is trivial $(x_{0}=x_{0}^{ol,x_{0}})$. The inductive step is verified from monotonicity: $x_{k+1}=f(x_{k},u_{k}^{ol,x_{0}},w_{k})\preceq f^{*}(x_{0}^{k},u_{k}^{k})=x^{ol,x_{0}}_{k+1}$. Therefore, ${\bf{{x}}}[0:h^{\varphi}]\preceq{\bf{{x}}}^{ol,x_{0}}[0:h^{\varphi}]$, $\forall{\bf{{w}}}[0:h^{\varphi}{-1}]\in\mathcal{W}^{h^{\varphi}}$. It follows from Lemma 1 that ${\bf{{x}}}[0:h^{\varphi}]\models\varphi,\forall{\bf{{w}}}[0:h^{\varphi}{-1}]\in\mathcal{W}^{h^{\varphi}}$. ∎

Corollary 1*.*

The set $\mathcal{X}_{0}^{\max}$ is a lower-set.

Proof:

Consider any $x_{0}^{\prime}\in L(x_{0}),x_{0}\in\mathcal{X}_{0}^{\max}$. Let $x^{\prime}_{k+1}=f(x^{\prime}_{k},u_{k}^{ol,x_{0}},w_{k}),k=0,1,\cdots,h^{\varphi}-1$. It follows from monotonicity that $x_{k}^{\prime}\preceq x_{k}^{ol,x_{0}},k=0,1,\cdots,h^{\varphi}$, $\forall{\bf{{w}}}[0:h^{\varphi}{-1}]\in\mathcal{W}^{h^{\varphi}}$. By the virtue of Lemma 1, ${\bf{{x}}}^{\prime}[0:h^{\varphi}]\preceq{\bf{{x}}}^{x_{0},ol}[0:h^{\varphi}]$. Therefore, we have $\forall x_{0}\in\mathcal{X}_{0}^{\max},x^{\prime}_{0}\in L(x_{0})\Rightarrow x^{\prime}_{0}\in\mathcal{X}_{0}^{\max}$, which indicates $\mathcal{X}_{0}^{\max}$ is a lower-set. ∎

Corollary 2*.*

If $x_{0}\in\mathcal{X}_{0}^{\max}$ and $\mu^{ol}$ is the following open-loop control policy

[TABLE]

then ${\bf{{x}}}(x^{\prime}_{0},\mu,{\bf{{w}}})[0:h^{\varphi}]\models\varphi,\forall{\bf{{w}}}\in\mathcal{W}^{h^{\varphi}},\forall x^{\prime}_{0}\in L(x_{0})$.

Proof:

Follows from the proof of Corollary 1. ∎

Now that we have established the properties of the solutions to Problem 1, we explain how to compute the admissible initial conditions and their corresponding open-loop control sequences. The approach is based on formulating the conditions in Theorem 1 as a set of constraints that can be incorporated into a feasibility solver. We convert all the constraints into a set of mixed-integer linear constraints and use off-the-shelf MILP solvers to check for feasibility. Converting logical properties into mixed-integer constraints is a common procedure which was employed for MLD systems in [7]. The authors in [8] and [10] extended this technique to a framework for time bounded model checking of temporal logic formulas. A variation of this method is explained here.

First, the STL formula is recursively translated into a set of mixed-integer constraints. For each predicate $\pi=(a_{\pi}^{T}x\leq b_{\pi})$, as in (7), we define a binary variable $z^{\pi}_{k}\in\{0,1\}$ such that 1 (respectively, 0) stands for true (respectively, false). The relation between $z^{\pi}$, robustness $\rho$, and $x$ is encoded as:

[TABLE]

The constant $M$ is a sufficiently large number such that $M\geq\max\{a_{\pi}^{T}K,b_{\pi}\}$, where $K\in\mathbb{R}^{n}_{+}$ is the upper bound for the state values, $x_{k}\preceq K,k=0,1,\cdots,h^{\varphi}$. In practice, $M$ is chosen sufficiently large such that the constraint $x\preceq K$ is never active. Note that the largest value of $\rho$ for which $z^{\pi}=1$ is $b_{\pi}-a^{T}_{\pi}x$, which is equal to the robustness of $\pi$.

Now we encode the truth table relations. For instance, we desire to capture $1\wedge 0=0$ and $1\vee 0=1$ using mixed-integer linear equations. Disjunction and conjunction connectives are encoded as the following constraints:

[TABLE]

where $z\in[0,1]$ is declared as a continuous variables. However, it only can take binary values as evident from (12). Similarly, define $z^{\varphi}_{k}\in[0,1]$ as the variable indicating whether ${\bf{{x}}}[k]\models\varphi$. An STL formula is recursively translated as:

[TABLE]

Finally, we add the following constraints:

[TABLE]

Proposition 1*.*

The set of constraints in (11),(12),(13),(14) has the following properties:

• i)

we have ${\bf{{x}}}[0]\models\varphi$ if the set of constraints is feasible;

• ii)

we have ${\bf{{x}}}[0]\not\models\varphi$ if the set of constraints is infeasible;

• iii)

the largest $\rho$ such that the set of constraints, while “$\rho\geq 0$” is removed from (14), is feasible is equal to $\rho({\bf{{x}}},\varphi,0)$.

Proof.

i) We provide the proof for (12), as the case for more complex STL formulas are followed in a recursive manner from (13). If $z=1$, we have from (12a) that $z_{i}=1,i=1,\cdots,n_{z}$, which correctly encodes conjunctions. Similarly, $z=1$ in (12b) indicates that not all $z_{i},i=0,\cdots,n_{z}$ can be zero, or, $\exists i\in\{1,\cdots,n_{z}\}$ such that $z_{i}=1$, which correctly encodes disjunctions. ii) Infeasibility can be recursively traced back into (12). For both (12a) and (12b), if $z=1$ is infeasible, it indicates that $z_{i}=0,i=1,\cdots,n_{z}$. iii) We also prove this statement for (12) as it is the base of recursion for general STL formulas. Let $z_{i}=(a^{T}_{\pi_{i}}x+\rho\leq b_{\pi_{i}}),i=1,\cdots,n_{z}$. Consider (12a) and the following optimization problem:

[TABLE]

where its solution is $\underset{i=1,\cdots,n_{z}}{\min}(b_{\pi_{i}}-a^{T}_{\pi_{i}}x)$, which is identical to the quantitative semantics for conjunction (see (4)). Similarly, consider (12b) and the following optimization problem:

[TABLE]

where the solution is $\underset{i=1,\cdots,n_{z}}{\max}(b_{\pi_{i}}-a^{T}_{\pi_{i}}x)$, which is identical to the quantitative semantics for disjunction. ∎

Our integer formulation for Boolean connectives slightly differs from the formulation in [8], [10], where lower bound constraints for the $z$’s are required. For example, for translating $z=\bigwedge_{i=1}^{n_{z}}z_{i}$, it is required to add $z\geq\sum_{i=1}^{n_{z}}z_{i}-n_{z}+1$ to impose a lower bound for $z$. However, these additional constraints become necessary only when the negation operator is present in the STL formula. Hence, they are removed in our formulation. This reduces the constraint redundancy and degeneracy of the problem. By doing so, we observed computation speed gains (up to reducing the computation time by 50%) in our case studies. Moreover, we encode quantitative semantics in a different way than [10], where a separate STL robustness-based encoding is developed which introduces additional integers. Due to property “iii” in Proposition 1, our encoding does not require additional integers to capture robustness hence it is computationally more efficient.

Definition 6*.*

System (5) is in MLD form [7] if written as:

[TABLE]

where $\delta_{t}\in\{0,1\}^{n_{\delta}}$ and $r_{t}\in\mathbb{R}^{n_{r}}$ are auxiliary variables and $A,B_{u},B_{w},D_{\delta},D_{r},E_{\delta},E_{r},E_{x},E_{u},E_{w},e$ are appropriately defined constant matrices such that (15) is well-posed in the sense that given $x_{t},u_{t},w_{t}$, the feasible set for $x_{t+1}$ is a single point equal to $f(x_{t},u_{t},w_{t})$. Introducing auxiliary variables and enforcing (15b) can capture nonlinear $f$ [7].

The system equations are brought into mixed-integer linear constraints by transforming system (5) into its MLD form. As mentioned earlier, any piecewise affine system can be transformed into an MLD. In the case studies of this paper, the construction of (15) from a piecewise affine (5) is not explained as the procedure is well documented in [40].

Finally, the set of constraints in Theorem 1 can be cast as:

[TABLE]

Checking the satisfaction of the set of constraints in (16) can be formulated as a MILP feasibility problem, which is handled using powerful off-the-shelf solvers. For a fixed initial condition $x_{0}$, the feasibility of the MILP indicates whether $x_{0}\in\mathcal{X}_{o}^{\max}$. An explicit representation of $\mathcal{X}_{o}^{\max}$ requires variable elimination from (16), which is computationally intractable for a large MILP. Alternatively, we can set $x_{0}$ as a free variable while maximizing a cost function (e.g. norm of $x_{0}$) such that a large $L(x_{0})$ is obtained. Another natural candidate is maximizing $\rho({\bf{{x}}}^{ol,x_{0}},\varphi,0)$. It is worth to note that by finding a set of distinct initial conditions and taking the union of all $L(x_{0})$, we are able to find a representation for an under-approximation of $\mathcal{X}_{o}^{\max}$.

MILPs are NP-complete. The complexity of solving (16) grows exponentially with respect to the number of binary variables and polynomially with respect to the number of continuos variables. The number of binary variables in our framework is $\mathcal{O}\left(h^{\varphi}({n_{\pi}}+m_{b}+q_{b}+n_{\delta})\right)$ - $n_{\pi}$ is the number of predicates - and the number of continuous variables is $\mathcal{O}\left(h^{\varphi}(n+m_{r}+q_{r}+n_{r})\right)$. In other words, the exponential growth builds upon the intricacy of the specification and the number of modes demonstrated by the hybrid nature of the system. However, the complexity is polynomial with respect to the dimension of the state.

Example 2*.*

Consider the following switched system:

[TABLE]

where $x=(x_{[1]},x_{[2]})^{T}\in\mathbb{R}_{+}^{2}$, $u\in\mathcal{U}$ is the control input (switch), $\mathcal{U}=\{1,2\}$, and

[TABLE]

The (additive) disturbance $w$ is bounded to $L(w^{*})$, where $w^{*}=(1.5,1)^{T}$ and $\tau=0.1$. This system is the discrete-time version of $\dot{x}=A_{u}x+w$ with sample time $\tau$. Both matrices are Metzler (all off-diagonal terms are non-negative hence all the elements of its exponential are positive) and non-Hurwitz hence constant input results in unbounded trajectories. The system is desired to satisfy the following STL formula:

[TABLE]

where $p_{1}=\left((x_{[1]}\leq 1)\wedge(x_{[2]}\leq 5)\right)$ and $p_{2}=\left((x_{[1]}\leq 5)\wedge(x_{[2]}\leq 1)\right)$. In plain English, $\varphi$ states that “within 10 time units, the trajectory visits the box characterized by $p_{1}$ first and then the box corresponding to $p_{2}$” (see Fig. 1). We transformed this system into its MLD form (15). We formulated the constraints in (16) as a MILP and set the cost function to maximize $\left\|x_{0}\right\|_{\infty}$ and used the Gurobi 333www.gurobi.com MILP solver. The solution was obtained in less than 0.05 seconds on a 3GHz Dual Core MacBook Pro. We obtained $x_{0}=(2.82~{}2.82)^{T}$ and the following open-loop control sequence: $1~{}2~{}1~{}2~{}2~{}1~{}1~{}1~{}1~{}1$. By applying this control sequence, we sampled a trajectory of the original system $f$ with values of $w$ drawn from a uniform distribution over $L(w^{*})$. Both the trajectories of $f$ and $f^{*}$ satisfy the specification. The results are shown in Fig. 1.

V Infinite Horizon Semantics

In this section, we provide a solution to Problem 2. We show that the infinite-time property in (8) can be guaranteed using repetitive control sequences. First, we consider global specifications and extend the results from our previous work [21] in Sec. V-A. Next, we show how to find controls for bounded-global STL formulas (Problem 2) in Sec. V-B. Solution completeness is discussed in Sec. V-C.

V-A Global formulas: s-sequences and inductive invariance

Consider the global specification ${\bf{{G}}}_{[0,\infty]}\varphi$, where $\varphi$ is a bounded formula. We introduce some additional notation.

Definition 7*.*

Given a bounded STL formula $\varphi$ over predicates in the form (7), the language realization set (LRS) [41] is:

[TABLE]

Proposition 2*.*

The set $\mathcal{L}^{\varphi}$ is a lower-set.

Proof:

For all $x_{0}x_{1}\cdots x_{h^{\varphi}}\in\mathcal{L}^{\varphi}$ and $x^{\prime}_{0}x^{\prime}_{1}\cdots x^{\prime}_{h^{\varphi}}\preceq x_{0}x_{1}\cdots x_{h^{\varphi}}$, it follows from Lemma 1 that $x^{\prime}_{0}x^{\prime}_{1}\cdots x^{\prime}_{h^{\varphi}}\models\varphi$. Thus, $x^{\prime}_{0}x^{\prime}_{1}\cdots x^{\prime}_{h^{\varphi}}\in LRS(\varphi)$ hence $LRS(\varphi)$ is a lower set. ∎

It follows from the semantics of global operator in (2) that ${\bf{{x}}}\models{\bf{{G}}}_{[0,\infty]}\varphi$ is equivalent to ${\bf{{x}}}[t:t+h^{\varphi}]\in\mathcal{L}^{\varphi},\forall t\in\mathbb{N}.$

Definition 8*.*

A set {\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}{\Omega_{\mathcal{L}^{\varphi}}}}\subseteq\mathcal{L}^{\varphi} is a robust control invariant (RCI) set if:

[TABLE]

Satisfaction of ${\bf{{G}}}_{[0,\infty]}\varphi$ is accomplished by finding a RCI set in $\mathcal{L}^{\varphi}$. Note that unlike traditional definitions of RCI sets (e.g., [42]), where the set is defined in the state-space $\mathcal{X}$, our RCI set is defined in an augmented form of the state-space $\mathcal{X}^{h^{\varphi}}$. The language realization set can also be interpreted as the “safe” set in $(h^{\varphi}+1)$-length trajectory space. The maximal RCI set inside $\mathcal{L}^{\varphi}$ provides a complete solution to the set-invariance problem. The computation of maximal RCI set requires implementing an iterative fixed-point algorithm which is computationally intensive for MLD systems and non-convex sets (see [43, 44] for discussion). We use monotonicity to provide an alternative approach. The following result is a more general version of the one in [21].

Theorem 2*.*

Given a bounded formula $\varphi$, if there exists ${\bf{{x}}}^{s}[0:h^{\varphi}]\in\mathcal{L}^{\varphi}$, and a sequence of controls: $u^{s}_{0},\cdots,u^{s}_{T-1}$ - where $T$ is a positive integer determining the length of the sequence - such that:

${\bf{{x}}}^{s}[k:k+h^{\varphi}]\in\mathcal{L}^{\varphi},k=0,1,\cdots,T$, where $x^{s}_{h^{\varphi}+k+1}=f^{*}(x^{s}_{h^{\varphi}+k},u^{s}_{k})$, 2. 2.

${\bf{{x}}}^{s}[T:T+h^{\varphi}]\preceq{\bf{{x}}}^{s}[0:h^{\varphi}]$,

then the following set is a RCI set in $\mathcal{L}^{\varphi}$:

[TABLE]

Proof:

For any x^{\prime}_{0}x^{\prime}_{1}\cdots x^{\prime}_{h^{\varphi}}\in{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}{\Omega_{\mathcal{L}^{\varphi}}}}, there exists $i\in\{0,1\cdots,T-1\}$ such that $x^{\prime}_{0}x^{\prime}_{1}\cdots x^{\prime}_{h^{\varphi}}\in L({\bf{{x}}}^{s}[i:i+h^{\varphi}])$. On one hand, we have $x_{i+1}^{s}\cdots x_{i+h^{\varphi}}^{s}f^{*}(x_{i+h^{\varphi}}^{s},u_{i}^{s})\in\Omega_{\mathcal{L}^{\varphi}}$. On the other hand, we have $x^{\prime}_{1}\preceq x_{i+1}^{s},\cdots,x^{\prime}_{h^{\varphi}}\preceq x_{i+h^{\varphi}}^{s}$. By applying $u_{i}^{s}$, monotonicity implies

[TABLE]

And the proof is complete from the fact that $x_{i+1}^{s}\cdots x_{i+1+h^{\varphi}}^{s}\in\Omega_{\mathcal{L}^{\varphi}}$ for all $i\in\{0,1\cdots,T-1\}$. ∎

Corollary 3*.*

Let the conditions in Theorem 2 hold and ${\bf{{x}}}[t_{0}:t_{0}+h^{\varphi}]\in L({\bf{{x}}}^{s}[0:h^{\varphi}])$ for some $t_{0}\in\mathbb{N}$. Consider the following control sequence starting from time $t_{0}+h^{\varphi}$:

[TABLE]

i.e., $u^{s}_{t}=u^{s}_{\operatorname*{rem}(t-t_{0}-h^{\varphi},T)},t\geq t_{0}+h^{\varphi}$. Let $x_{k+1}=f^{*}(x_{k},u_{k}),k=t_{0}+h^{\varphi},\cdots$. Then we have ${\bf{{x}}}[t:t+h^{\varphi}]\in\Omega_{\mathcal{L}^{\varphi}},\forall t\geq t_{0}$.

Proof.

We prove by induction that {\bf{{x}}}[t:t+h^{\varphi}]\in L({\bf{{x}}}^{s}[\operatorname*{rem}(t-t_{0},T):\operatorname*{rem}(t-t_{0}{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}{,T}})+h^{\varphi}]),\forall t\geq t_{0}. The base case for $t=t_{0}$ is true. In order to prove the inductive step ${\bf{{x}}}[t+1:t+1+h^{\varphi}]\in L({\bf{{x}}}^{s}[\operatorname*{rem}(t+1-t_{0},T):\operatorname*{rem}(t+1-t_{0},T)+h^{\varphi}])$, we need to prove that $x_{t+k+1}\preceq x^{s}_{\operatorname*{rem}(t+1-t_{0},T)+k},k=0,\cdots,h^{\varphi}$, for which we need to only prove the case for $k=h^{\varphi}$ as previous inequalities are already assumed by induction. We show $x_{t+h^{\varphi}+1}\preceq x^{s}_{\operatorname*{rem}(t+1-t_{0},T)+h^{\varphi}}$ through monotonicity and the induction assumption that $x_{t+h^{\varphi}}\preceq x^{s}_{\operatorname*{rem}(t-t_{0},T)+h^{\varphi}}$:

[TABLE]

Note that $x^{s}_{T+h^{\varphi}}\preceq x^{s}_{h^{\varphi}}$. The “$\preceq$” in the last line can be replaced by “$=$” when $\operatorname*{rem}(t-t_{0},T)+1\neq T$. ∎

We refer to the repetitive sequence of controls in (20) as an s-sequence. An s-sequence is an invariance inducing open-loop control policy. Once the latest $h^{\varphi}+1$-length of system state are brought into $\Omega_{\mathcal{L}^{\varphi}}$, an s-sequence keeps the $h^{\varphi}+1$-length trajectory of the system in $\Omega_{\mathcal{L}^{\varphi}}$ for all subsequent times.

The computation of an s-sequence requires solving an MILP for ${\bf{{x}}}^{s}[h^{\varphi}:T]\models{\bf{{G}}}_{[0,T]}\varphi$ (an instance of Problem 1) with an additional set of constraints in ${\bf{{x}}}^{s}[0:h^{\varphi}]\models\varphi$ (again, an instance of Problem 1, but without the dynamical constraints. In other words, ${\bf{{x}}}^{s}[0:h^{\varphi}]$ does not need to be a trajectory of the maximal system), and ${\bf{{x}}}^{s}[T:T+h^{\varphi}]\preceq{\bf{{x}}}^{s}[0:h^{\varphi}]$ (linear constraints). We are usually interested in the shortest s-sequence since its computation requires the smallest MILP. Algorithmically, we start from $T=1$ and implement $T\leftarrow T+1$ until the MILP formulating the conditions in Theorem 2 becomes feasible and an s-sequence is found. As it will be implied from results in Sec. V-C, existence of an s-sequence is almost necessary for existence of a RCI set.

Example 3*.*

Consider the system in Example 2. We wish to keep the trajectory in the set characterized by $p_{1}\vee p_{2}$, i.e., $\mathcal{S}=L\left((1,5)^{T})\right)\cup L\left((5,1)^{T})\right)$. Note that this set is non-convex. We set the cost function to maximize $\left\|x_{0}\right\|_{1}$. The shortest s-sequence has $T=5$ and is: $(2~{}1~{}2~{}1~{}1)^{\omega}$. The resulting trajectory satisfying the definition of s-sequence is shown in Fig. 2 (a). The corresponding robust control invariant set is shown in Fig. 2. (b) (cyan region), which is characterized by the $x^{s}_{0},x^{s}_{1},\cdots,x^{s}_{4}$ (red dots) that lie inside $\mathcal{S}$ (green region). Note that the $[0,2]\times[0,2]$ portion of the coordinates in Fig. 1 is shown here for a clearer representation of the details.

V-B Bounded-global specifications: $\phi$-sequences

Now we consider general bounded-global formulas - as in Problem 2 - and generalize the paradigm used for s-sequences. We provide the key result of this section.

Theorem 3*.*

Given a bounded-global STL formula $\phi=\varphi_{b}\wedge{\bf{{G}}}_{[\Delta,\infty]}\varphi_{g}$, an initial condition $x_{0}$, a control sequence $u_{0}^{\phi}\cdots u_{\Delta+T+h^{\varphi_{g}}-1}^{\phi}$, where $T$ is a positive integer, and a non-negative integer $T_{0}<T$, let the following conditions hold:

${\bf{{x}}}^{\phi}[0:\Delta+T+h^{\varphi_{g}}]\models\varphi_{b}\wedge{\bf{{G}}}_{[\Delta,\Delta+T]}\varphi_{g}$, 2. 2.

${\bf{{x}}}^{\phi}[\Delta+T:\Delta+T+h^{\varphi_{g}}]\preceq{\bf{{x}}}^{\phi}[\Delta+T_{0}:\Delta+T_{0}+h^{\varphi_{g}}]$;

where $x_{k+1}^{\phi}=f^{*}(x_{k}^{\phi},u_{k}^{\phi}),k\in[0,\Delta+T+h^{\varphi}_{g}-1]$, $x_{0}^{\phi}=x_{0}$. Let $\mu^{ol}$ be the open-loop control policy corresponding to the following control sequence:

[TABLE]

Then ${\bf{{x}}}(x^{\prime}_{0},\mu^{ol},{\bf{{w}}})\models\phi,\forall{\bf{{w}}}\in\mathcal{W}^{\omega},\forall x^{\prime}_{0}\in L(x_{0}).$ Moreover, the following set is a RCI set in $\mathcal{L}^{\varphi_{g}}$:

[TABLE]

Proof:

We need to prove that ${\bf{{x}}}(x^{\phi}_{0},\mu^{ol},{\bf{{w}}}^{*})\models\phi$, where ${\bf{{w}}}^{*}=(w^{*})^{\omega}$. The fact that ${\bf{{x}}}(x^{\prime}_{0},\mu^{ol},{\bf{{w}}})[0]\models\phi,\forall x^{\prime}_{0}\in L(x_{0}),\forall{\bf{{w}}}\in\mathcal{W}^{\omega}$, follows from monotonicity and Lemma 1. The fact that $\Omega_{\mathcal{L}^{\varphi_{g}}}$ is a RCI set follows from Theorem 2 as (19) is obtained from replacing $\Delta=T_{0}=0$ in (22) . It follows that $\left(u_{\Delta+T_{0}+h^{\varphi_{g}}}^{\phi}\cdots u_{\Delta+T+h^{\varphi_{g}}-1}^{\phi}\right)^{\omega}$ is an s-sequence. For all $t\geq\Delta+T+h^{\varphi_{g}}$, let

[TABLE]

Using Corollary 3, we have ${\bf{{x}}}^{\phi}[k+\Delta+T_{0}:k+\Delta+T_{0}+h^{\varphi_{g}}]\in\mathcal{L}^{\varphi_{g}},\forall k\in\mathbb{N}$, and the proof is complete.

∎

We refer to the sequence of controls in (21) as a $\phi$-sequence. The computation of a $\phi$-sequence requires solving an MILP for ${\bf{{x}}}^{\phi}[0:\Delta+T+h^{\varphi_{g}}]\models\varphi_{b}\wedge{\bf{{G}}}_{[\Delta,\Delta+T]}\varphi_{g}$ (an instance of Problem 1) with an additional set of constraints in ${\bf{{x}}}^{\phi}[\Delta+T:\Delta+T+h^{\varphi_{g}}]\preceq{\bf{{x}}}^{\phi}[\Delta+T_{0}:\Delta+T_{0}+h^{\varphi_{g}}]$ (linear constraints). Thus, similar to s-sequecnes, the computation of a $\phi$-sequence is based on feasibility checking of a MILP. We have two parameters $T$ and $T_{0}<T$ to search over. We start from $T=1$ and implement $T\leftarrow T+1$, while checking for all $T_{0}<T$, until the corresponding MILP gets feasible. In Sec. V-C, we discuss the necessity of existence of a feasible solution for some $T_{0},T$.

Another interpretation of a $\phi$-sequence is a sequence that consists of an initialization segment of length $\Delta+h^{\varphi_{g}}$ to bring the latest $h^{\varphi_{g}}$ states of the system into $\Omega_{\mathcal{L}^{\varphi_{g}}}\subseteq\mathcal{L}^{\varphi_{g}}$ and a repetitive segment of length $T$ to stay in $\Omega_{\mathcal{L}^{\varphi_{g}}}$. The repetitive segment is an s-sequence. Since control inputs eventually becoming periodic, the long-term behavior is expected to demonstrate periodicity, which leads to the following result based on Theorem 3.

Corollary 4*.*

The $\omega$-limit set of the run given by (23) is non-empty and corresponds to the following periodical orbit:

[TABLE]

where $x_{k}^{\phi,\infty}:=\lim_{c\rightarrow\infty}x_{k+\Delta+T_{0}+c(T-T_{0})}^{\phi},k=0,\cdots,T-T_{0}-1.$

Proof:

We show that $x^{\phi}_{t}\preceq x^{\phi}_{t+T-T_{0}},\forall t\geq\Delta+T_{0}$. Similar to the proof of Corollary 3, we use induction. The base case for $t=\Delta+T_{0}$ is already in the second condition in Theorem 3. The inductive step is proven as follows:

[TABLE]

where from (21) we have

[TABLE]

Thus, each component of the sequence $x^{\phi}_{\Delta+T_{0}+k}x^{\phi}_{\Delta+T+k}x^{\phi}_{\Delta+2T-T_{0}+k}\cdots$, $k=0,\cdots,T-T_{0}$, is monotonically decreasing. Monotone convergence theorem [45] explains that a lower-bounded monotonically decreasing sequence converges (in this case, all values are lower-bounded by zero). Thus, $\lim_{c\rightarrow\infty}x_{\Delta+T_{0}+k+c(T-T_{0})}^{\phi},k=0,\cdots,T-T_{0},$ exists and the proof is complete. ∎

Example 4*.*

Consider the system in Example 2. We wish to satisfy

[TABLE]

The specification is in form in (9) with $h^{\varphi_{b}}=\Delta=5,h^{\varphi}_{g}=6$. This specification requires that $p_{1}$ is visited at least once until $t=5$ and, afterwards, $p_{1}$ and $p_{2}$ are persistently visited while the maximum time between two subsequent visits is not greater than $6$. We find a $\phi$-sequence solving a MILP for $T=7,T_{0}=0,$ while maximizing $\left\|x_{0}\right\|_{1}$. The obtained $\phi$-sequence is ${\bf{{u}}}^{\phi}=2~{}2~{}2~{}2~{}1~{}2~{}1~{}1~{}1~{}1~{}2~{}(2~{}2~{}1~{}1~{}1~{}1~{}2)^{\omega}$ for $x_{0}=(12.4,0)^{T}$. The first $\Delta+T+h^{\varphi_{g}}+1=5+7+6+1=19$ time points of the trajectory of the maximal system $f^{*}$ satisfying the conditions in Theorem 3 are shown in Fig. 3 [Left]. A sample trajectory of $f$ with values of ${\bf{{w}}}$ chosen uniformly from $\mathcal{W}$ is also shown. Both trajectories satisfy $\phi$. The limit-set of $f^{*}$, which is a 7-periodical orbit, is shown in Fig. 3 [Right].

V-C Necessity of Open-loop Strategies

We showed that if there exists an initial condition and a finite length control sequence such that the statements in Theorem 2 hold, an open-loop control sequence is sufficient for satisfying of a bounded-global formula, as was formulated in Problem 2. In this section, we address the necessity conditions.

We show that the existence of open-loop control strategies for satisfying a bounded-global specifications is almost necessary in the sense that if a $\phi$-sequence is not found using Theorem 3 for large values of $T$, then it is almost certain that no correct control policy (including feedback policies) exists, or, if exists any, it is fragile in the sense that a slight increase in the effect of the disturbances makes the policy invalid. We characterize the necessity conditions based on hypothetical perturbations in the disturbance set.

Theorem 4*.*

Suppose system (5) is strongly monotone with respect to the maximal disturbance in the sense that for all $\epsilon>0$, there exists a perturbed disturbance set $\mathcal{W}_{p}$ with maximal disturbance $w^{*}_{p}$ such that

[TABLE]

Consider the bounded-global formula $\phi=\varphi_{b}\wedge{\bf{{G}}}_{[\Delta,\infty]}\varphi$. Given $\epsilon>0$, the disturbance set is altered to $\mathcal{W}_{p}$ such that (25) holds. If there exists a control policy $\mu$ and an initial condition $x_{0}$ such that ${\bf{{x}}}(x_{0},\mu,{\bf{{w}}}_{p})\models\phi,\forall{\bf{{w}}}_{p}\in\mathcal{W}_{p}^{\omega}$, then there exists at least one open-loop control policy $\mu^{ol}$ in the form of a $\phi$-sequence in (21) for the original system such that

[TABLE]

where $A$ is a constant depending on $\mathcal{L}^{\varphi_{g}}$.

Proof:

Given a bounded set $\mathcal{C}\subset\mathbb{R}^{n(h^{\varphi}+1)}$, we define the diameter $d(\mathcal{C}):=\inf\{d|s_{1}\preceq s_{2}+d{1}_{n(h^{\varphi}+1)},\forall s_{1},s_{2}\in\mathcal{C}\}$ (e.g., the diameter of an axis-aligned hyper-box is equal to the length of its largest side). Consider a partition of $\mathcal{L}^{\varphi}$ by a finite number of cells, where the diameter of each cell is less than $\epsilon$. The maximum number of cells required for such a partition is $\nicefrac{{A}}{{\epsilon^{n(h^{\varphi_{g}}+1)}}}$, where $A$ is a constant dependent on the shape and volume of $\mathcal{L}^{\varphi_{g}}$. A conservative upper bound on $A$ can be given as follows. Define $a^{*}\in\mathbb{R}_{+}$ as

[TABLE]

Since $\mathcal{L}^{\varphi_{g}}$ is bounded and closed, $a^{*}$ exists. We have $\mathcal{L}^{\varphi_{g}}\subseteq L(a^{*}1_{n(h^{\varphi_{g}}+1)})$. Let $A$ be ${a^{*}}^{n(h^{\varphi_{g}}+1)}$ - the volume of $L(a^{*}1_{n(h^{\varphi_{g}}+1)})$, which is a hyper-box. Partition $L(a^{*}1_{n(h^{\varphi_{g}}+1)})$ into $N:=\nicefrac{{A}}{{\epsilon^{n(h^{\varphi_{g}}+1)}}}$ number of equally sized cubic cells with side length of $\epsilon$. Such a partition also partitions $\mathcal{L}^{\varphi_{g}}$ to at most $N$ number of cells where the diameter of each cell is not greater than $\epsilon$.

Since there exists $\mu$ such that ${\bf{{x}}}(x_{0},\mu,{\bf{{w}}}_{p})\models\phi,\forall{\bf{{w}}}_{p}\in\mathcal{W}_{p}^{\omega}$, then there exist at least one run satisfying $\phi$ for system $x_{k+1}=f(x_{k},u_{k},w_{p}^{*})$. Let $x_{0},\cdots,x_{\Delta+h^{\varphi_{g}}+N}$ be the first $\Delta+h^{\varphi_{g}}+N+1$ time points of a such a run. We have ${\bf{{x}}}[k:k+h^{\varphi_{g}}]\in\mathcal{L}^{\varphi_{g}},k=\Delta,\cdots,\Delta+N$. Consider the sequence ${\bf{{x}}}[\Delta:\Delta+h^{\varphi_{g}}]{\bf{{x}}}[\Delta+1:\Delta+1+h^{\varphi_{g}}]\cdots{\bf{{x}}}[\Delta+N:\Delta+N+h^{\varphi_{g}}]$. Consider a partition of $\mathcal{L}^{\varphi_{g}}$ with cells that for all cells the diameter is less than $\epsilon$. By the virtue of pigeonhole principle, there exists a cell that contains at least two time points ${\bf{{x}}}[{k_{1}}:k_{1}+h^{\varphi_{g}}]$ and ${\bf{{x}}}[{k_{2}}:k_{2}+h^{\varphi_{g}}]$, $\Delta\leq k_{1}\leq k_{2}\leq\Delta+N$. From the assumption on the diameter of the cells we have

[TABLE]

Now consider system $x^{\prime}_{k+1}=f(x^{\prime}_{k},u_{k},w^{*})$ - the original maximal system - with $x^{\prime}_{k_{1}+h^{\varphi_{g}}}=x_{k_{1}+h^{\varphi_{g}}}$. We prove that

[TABLE]

We use induction. The base case for $k=k_{1}+h^{\varphi_{g}}+1$ is verified using (25):

[TABLE]

The inductive step is verified using monotonicity and (25):

[TABLE]

It immediately follows from (28) that

[TABLE]

Since the lefthand of (27) is the righthand of (29), we have:

[TABLE]

This is reminiscent of the conditions in Theorem 2. Now by defining $x^{\prime}_{k}:=x_{k},k=k_{1},\cdots,k_{1}+h^{\varphi_{g}}-1$, we conclude that

[TABLE]

is a RCI set for system with adversarial disturbance set $\mathcal{W}$ and $(u_{k_{1}}\cdots u_{k_{2}+h^{\varphi}_{g}-1})^{\omega}$ is an s-sequence.

Now, once again, consider the original system $x^{\prime}_{k+1}=f(x^{\prime}_{k},u_{k},w^{*})$ with $x^{\prime}_{0}=x_{0}$. Monotonicity implies ${\bf{{x^{\prime}}}}[0:k_{1}+h^{\varphi_{g}}]\leq{\bf{{x}}}[0:k_{1}+h^{\varphi_{g}}]$. Thus, by applying $u_{0},\cdots,u_{k_{1}+h^{\varphi_{g}}-1}$ and using Lemma 1, we have ${\bf{{x^{\prime}}}}[0:k_{1}+h^{\varphi_{g}}]\models\varphi_{b}\wedge{\bf{{G}}}_{[\Delta,k_{1}]}\varphi_{g}$. Corollary 3 implies $x^{\prime}[k_{1}+h^{\varphi_{g}}]\models{\bf{{G}}}_{[0,\infty)}\varphi_{g}$ if $(u_{k_{1}},\cdots,u_{k_{2}-1})^{\omega}$ is applied starting from time $k_{1}$. Finally, monotonicity and Lemma 1 immediately indicate that ${\bf{{x}}}(x^{\prime\prime}_{0},\mu^{ol},{\bf{{w}}})\models\phi,\forall x^{\prime\prime}_{0}\in L(x_{0}),\forall{\bf{{w}}}\in\mathcal{W}^{\omega}$, where $\mu^{ol}$ is the following open-loop control strategy producing the following control sequence:

[TABLE]

which is in form of (21) with $k_{1}=\Delta+T_{0}+h^{\varphi_{g}}$ and $k_{2}=\Delta+T$. Since $k_{2}\leq\Delta+N$, we also have $T\leq N,N=\nicefrac{{A}}{{\epsilon^{n(h^{\varphi_{g}}+1)}}}$, and the proof is complete.

∎

Corollary 5*.*

Suppose that for all $T\leq T^{\max},T_{0}<T$, there does not exist an initial condition and a control sequence such that the conditions in Theorem 3 hold. Then there does not exist any solution to Problem 2 given that the maximal disturbance is $w_{p}^{*}$ such that (25) holds with $\epsilon>\sqrt[n(h^{\varphi_{g}}+1)]{T^{\max}}$.

The relation between the fragility in Theorem 4 and the length of the $\phi$-sequence suggests that by performing the search for longer $\phi$-sequences (which are computationally more difficult), the bound for fragility becomes smaller, implying that a correct control policy (if exists) is close to the limits (i.e., robustness score is close to zero, or the constraints are barely satisfied in the case with maximal disturbance). In practice, the bounds in Theorem 4 are very conservative and one may desire to find tighter bounds for specific applications.

Example 5*.*

Consider Example 3. Suppose that there does not exist an s-sequence of length smaller than 144 with maximal disturbance $w^{*}$. The constant $A$ (area in this 2D case, see proof of Theorem 4) of region corresponding to $p_{1}\vee p_{2}$ is $9$. Therefore, $\mathcal{S}$ can be partitioned into $144$ equally sized square cells with side length $0.25$. Note that we have $\epsilon^{2}\geq 9/T$. Since the disturbances are additive, it follows that if $A_{u}^{-1}(I-e^{-A_{u}t})(w^{*}_{p}-w^{*})>(0.25,0.25)^{T},u=1,2$, then there does not exist any control strategy $\mu$ and $x_{0}\in\mathbb{R}_{+}^{n}$ such that ${\bf{{x}}}(x_{0},\mu,{\bf{{w}}}_{p})\models{\bf{{G}}}_{[0,\infty]}(x\in\mathcal{S}),\forall{\bf{{w}}}_{p}\in\mathcal{W}_{p}^{\omega}$.

VI Model Predictive Control

In this section, we provide a solution to Problem 3. We assume full knowledge of the history of state. As mentioned in Sec. III, the cost function $J$ is assumed to be non-decreasing with respect to the state values hence the system constraints are replaced with those of the maximal system. First, we explain the MPC setup for global STL formulas. Next, we prove that the proposed framework is guaranteed to generate runs that satisfy the global STL specification (8).

Let $t\geq h^{\varphi}-1$. The case of $t<h^{\varphi}-1$ is explained later. Given planning horizon $H$, the states that are predictable at time $t$ using controls in $u^{H}_{t}$ are $x_{1|t},x_{2|t},\cdots,x_{H|t}$. Given predictions $x_{1|t},x_{2|t},\cdots,x_{H|t}$, we need to enforce ${\bf{{x}}}[{t-h^{\varphi}+1,t+H}]\models{\bf{{G}}}_{[0,H-1]}\varphi$ at time $t$. Notice that

[TABLE]

i.e., the first $h^{\varphi}$ time points are actual values, the rest are predictions. Also, note that the values in ${\bf{{x}}}[\tau:\tau+h^{\varphi}]$ are independent of the values in $x_{t}^{H}$ for $\tau\leq t-h^{\varphi}$ and are not fully available for $\tau>t+H-h^{\varphi}$. Thus, $[t-h^{\varphi}+1,t+H-h^{\varphi}]$ is the time window for imposing constraints at time $t$ [12].

The MPC optimization problem is initially written as (we do not use it for control synthesis as explained shortly):

[TABLE]

The set of constraints in (32) requires the knowledge of $x_{t-h^{\varphi}+1}x_{t-h^{\varphi}+2}\cdots x_{t}$. Thus, the proposed control policy requires a finite memory for the history of last $h^{\varphi}$ states. As it will be shown in Proposition 3, persistent feasibility of the constraints in (32) leads to fulfilling ${\bf{{G}}}_{[0,\infty]}\varphi$. However, persistent feasibility of the MPC setup in (32) is not guaranteed. We address this issue for the remainder of this section.

Definition 9*.*

An MPC strategy is recursively feasible if, for all $t\in\mathbb{N}$, the control at time $t$ is selected such that the MPC optimization problem at $t+1$ becomes feasible.

Our goal is to modify (32) such that it becomes recursively feasible. It is known that adding a (the maximal) RCI set acting as a terminal constraint is sufficient (and necessary) to guarantee recursive feasibility [46]. We add the terminal constraint ${\bf{{x}}}[t+H-h^{\varphi}:t{+H}]\in\Omega_{\mathcal{L}^{\varphi}}$ to (32) to obtain:

[TABLE]

Proposition 3*.*

Let $\mu_{t}(x_{0},\cdots,x_{t})=\mu_{t}(x_{t-h^{\varphi}+1},\cdots,x_{t})=u^{H,opt}_{0|t}$, where $u^{H,opt}=u^{H,opt}_{0|t}\cdots u^{H,opt}_{H-1|t}$ is given by (33). If the optimization problem (33) is feasible for all $t\geq h^{\varphi}-1$, then ${\bf{{x}}}(x_{0},\mu,{\bf{{w}}})[0]\models{\bf{{G}}}_{[0,\infty]}\varphi,\forall{\bf{{w}}}\in\mathcal{W}^{\omega}$.

Proof.

We show that ${\bf{{x}}}(x_{0},\mu,{\bf{{w}}})[0:k+h^{\varphi}]\models{\bf{{G}}}_{[0,k]}\varphi,\forall{\bf{{w}}}\in\mathcal{W}^{*},\forall k\in\mathbb{N},$ using induction over $k$. Consider (33) for $t=k+h^{\varphi}-1$ for any $k\in\mathbb{N}$. The second constraint in (33) requires ${\bf{{x}}}_{|t}[k,k+h^{\varphi}]\models\varphi$, or equivalently, $x_{k}\cdots x_{k+h^{\varphi}-1}x_{1|k+h^{\varphi}-1}\models\varphi$. By applying $u^{opt}_{0|t}$, monotonicity implies $x_{k+h^{\varphi}}=f(x_{k+h^{\varphi}-1},u^{opt}_{0|t},w)\preceq x_{1|k+h^{\varphi}-1}=f^{*}(x_{k+h^{\varphi}-1},u^{opt}_{0|t}),\forall w\in\mathcal{W}.$ From Lemma 1 we have ${\bf{{x}}}[k:k+h^{\varphi}]\models\varphi$. Thus, we have shown ${\bf{{x}}}[k:k+h^{\varphi}]\models\varphi,\forall k\in\mathbb{N}$, and the proof is complete. ∎

Proposition 4*.*

The MPC strategy corresponding to (33) is recursively feasible.

Proof:

Suppose $u_{t}^{H}=u_{0|t}\cdots u_{H-1|t}$ and $x_{t}^{H}=x_{t+1|t}\cdots,x_{t+H-1|t}$ is a feasible solution for (33) at time $t$. Since $\Omega_{\mathcal{L}^{\varphi}}$ is a RCI set, there exist $u^{r}\in\mathcal{U}$ such that ${\bf{{x}}}_{|t}[{t+H+1}-h^{\varphi}:t+H+1]=x_{H-h^{\varphi}+1|t}x_{H-h^{\varphi}+2|t}\cdots x_{H|t}f(x_{H|t},u^{r},w)\in\Omega_{\mathcal{L}^{\varphi}},\forall w\in\mathcal{W}$. Suppose $u_{0|t}$ is applied to the system. We have $x_{t+1}=f(x_{t},u_{0|t},w)\preceq f^{*}(x_{t},u_{0|t})=x_{1|t},\forall w\in\mathcal{W}$.

Now, we prove that the optimization problem at time $t+1$ is feasible by showing that at least one feasible solution exists. Let $u^{H}_{t+1}=u_{1|t}u_{2|t}\cdots,u_{H|t}u^{r}$. We already showed that $x_{t+1}=x_{0|t+1}\preceq x_{1|t}$. By induction and using monotonicity, it follows that $x_{k|t+1}\preceq x_{k+1|t},k=,1,\cdots,H-2$. Therefore, we have $x_{t-h^{\varphi}+2}\cdots x_{t+1}x_{1|t+1}\cdots x_{H-1|t+1}\preceq x_{t-h^{\varphi}+2}\cdots x_{1|t}x_{2|t}\cdots x_{H|t}$, which using Lemma 1 establishes $x_{t-h^{\varphi}+2}\cdots x_{t+1}x_{1|t+1}\cdots x_{H-1|t+1}\models{\bf{{G}}}_{[0,H-1]}\varphi$. In order to complete the proof, it remains to show that ${\bf{{x}}}[t+H+1-h^{\varphi}:t+H+1]=x_{H+1-h^{\varphi}|t}\cdots x_{H|t+1}\models\varphi$. This follows from invariance. Note that $x_{H|t+1}=f^{*}(x_{H|t},u^{r})$. Therefore $x_{H+1-h^{\varphi}|t}\cdots\cdots x_{H|t+1}\in\Omega_{\mathcal{L}^{\varphi}}$, and since $\Omega_{\mathcal{L}^{\varphi}}\in\mathcal{L}^{\varphi}$, we have $x_{H+1-h^{\varphi}|t}\cdots\cdots x_{H|t+1}\models\varphi$, and the proof is complete. ∎

The MPC optimization problem is also converted into a MILP problem. It is computationally easier to solve the optimization problem in (33) by solving $T$ MILPs:

[TABLE]

Note that all MILPs can be aggregated into a single large MILP in the expense of additional constraints for capturing non-convexities of the terminal condition.

Finally, consider $t<h^{\varphi}$. In this case, we require $H\geq h^{\varphi}$ and replace the interval $[t-h^{\varphi}+1,t+H-h^{\varphi}]$ with $[0,t+H-h^{\varphi}]$ for $t<h^{\varphi}$ in (34). For applications where initialization is not important in long-term (like traffic management), a simpler approach is to initialize the MPC from $t=h^{\varphi}-1$ and assume all previous state values are zero (hence all the past predicates are evaluated as true).

Remark 3*.*

In our previous work on STL MPC of linear systems [12], we did not establish recursive feasibility. In order to recover from possible infeasibility issues, we proposed maximizing the STL robustness score (a negative value) whenever the MPC optimization problem became infeasible. Although recursive feasibility is guaranteed here, un-modeled disturbances and initial conditions outside $\mathcal{X}_{0}^{\max}$ can lead to infeasibility. The formalism in [12] can be used to recover from infeasibility with minimal violation of the specification.

VII Application to Traffic Management

In this section, we explain how to apply our methods to traffic management. First, the model that we use for traffic networks is explained, which is similar to the one in [47] but freeways are also modeled. Next, the monotonicity properties of the model are discussed. We show that there exists a congestion-free set in the state-space in which the traffic dynamics is monotone. Finally, a case study on a mixed urban and freeway network is presented.

VII-A Model

The topology of the network is described by a directed graph $(\mathcal{V},\mathcal{L})$, where $\mathcal{V}$ is the set of nodes and $\mathcal{L}$ is the set of edges. Each $l\in\mathcal{L}$ represents a one-way traffic link from tail node $\tau(l)\in\mathcal{V}\cup\emptyset$ to head node $\eta(l)\in\mathcal{V}$, where $\tau(l)=\emptyset$ stands for links originating from outside of the network. We distinguish between three types of links based on their control actuations: 1) $\mathcal{L}_{r}$: road links actuated by traffic lights, 2) $\mathcal{L}_{o}$: freeway on-ramps actuated by ramp meters, 3) $\mathcal{L}_{f}$: freeway segments which are not directly controlled. Freeway off-ramps are treated the same way as the roads. Uncontrolled roads are also treated the same as freeways. We have $\mathcal{L}_{r}\cup\mathcal{L}_{o}\cup\mathcal{L}_{f}=\mathcal{L}$.

Remark 4*.*

Some works, e.g. [14], consider control over freeway links by varying speed limits, which adds to the control power but requires the existence of such a control architecture within the infrastructure. We do not consider this type of control actuation in this paper but it can easily be incorporated into our model by modeling freeways links the same way as on-ramps, where the speed limit becomes analogous to the ramp meter input.

The number of vehicles on link $l$ at time $t$ is represented by $x_{[l],t}\in[0,c_{l}]$, which is assumed to be a continuous variable, and $c_{l}$ is the capacity of $l$. In other words, vehicular movements are treated as fluid-like flow in our model. The number of vehicles that are able to flow out of $l$ in one time step, if link $l$ is actuated, is:

[TABLE]

where $\bar{q}_{l}$ is the maximum outflow of link $l$ in one time step, which is physically related to the speed of the vehicles. The last argument in the minimizer determines the minimum supply available in the downstream links of $l$, where $\alpha_{l:l^{\prime}}\in[0,1]$ is the capacity ratio of link $l^{\prime}$ available to vehicles arriving from link $l$ (typically portion of the lanes), $\beta_{l:l^{\prime}}\in[0,1]$ is the ratio of the vehicles in $l$ that flow into $l^{\prime}$ (turning ratio). For simplicity, we assume capacity ratios and turning ratios are constants. System state is represented by $x\in\mathbb{R}^{n}_{+}:\{x_{[l]}\}_{l\in\mathcal{L}}$, where $n$ is the number of the links in the network. The state space is $\mathcal{X}:=\prod_{l\in\mathcal{L}}[0,c_{l}].$

A schematic diagram illustrating the behavior of $q_{[l]}$ with respect to the state variables $x_{[l]},x_{[l^{\prime}]}$- which is known as the fundamental diagram in the traffic literature [48] - is shown in Fig. 4. The link flow drops if one (or more) of its downstream links do not have enough capacity to accommodate the incoming flow. In this case (when the last argument in (35) is the minimizer), we say the traffic flow is congested. Otherwise, the traffic flow is free. This motivates the following definition:

Definition 10*.*

The congestion-free set, denoted by $\Pi$, is defined as the following region in the state space:

[TABLE]

Proposition 5*.*

The congestion-free set is a lower-set.

Proof:

Consider $x\in\Pi$ and any $x^{\prime}\in L(x)$. For all $l,{l^{\prime}}\in\mathcal{L},\tau(l^{\prime})=\eta(l)$, we have $\min\{x^{\prime}_{l},\bar{q}_{l}\}\leq\min\{x_{[l]},\bar{q}_{l}\}$ and $(c_{l^{\prime}}-x_{[l]^{\prime}})\leq(c_{l^{\prime}}-x^{\prime}_{l^{\prime}})$. Therefore, $\min\{x^{\prime}_{l},\bar{q}_{l}\}\leq\frac{\alpha_{l:l^{\prime}}}{\beta_{l:l^{\prime}}}(c_{l^{\prime}}-x^{\prime}_{l^{\prime}})$. Thus $x^{\prime}\in\Pi$, which indicates $\Pi$ is a lower-set. ∎

Note that $\Pi$ is, in general, non-convex. The predicate $(x\in\Pi)$ can be written as a Boolean logic formula over predicates in the form of (7) as:

[TABLE]

Notice how the minimizer in (36) is translated to a disjunction in (37).

Now we explain the controls. The actuated flow of link $l$ at time $t$ is denoted by $\vec{q}_{[l],t}$, where we have the following relations:

[TABLE]

where $s_{[l],t}\in\{0,1\}$ is the traffic light for link $l$, where $1$ (respectively, [math]) stands for green (respectively, red) light, and $r_{[l],t}\in\mathbb{R}_{+}$ is the ramp meter input for on-ramp $l$ at time $t$. Ramp meter input limits the number of vehicles that are allowed to enter the freeway in one time step. In order to disallow simultaneous green lights for links $l,l^{\prime}$ (which are typically pair of links pointing toward a common intersection in perpendicular directions), we add the additional constraints $s_{[l],t}+s_{[l]^{\prime},t}\leq 1$. In simple gridded networks, as in our case study network illustrated in Fig. 5, it is more convenient to define phases for actuation in north-south or east-west directions that are unambiguously mapped to traffic lights for each individual link. The evolution of the network is given by:

[TABLE]

where $w_{[l],t}$ is the number of exogenous vehicles entering link $l$ at time $t$, which is viewed as the adversarial input. The evolution relation above can be compacted into the form (5):

[TABLE]

where $u_{t}$ and $w_{t}$ are the vector representations for control inputs (combination of traffic lights and ramp meters) and disturbances inputs, respectively. Note that $f_{\text{traffic}}$ represents a hybrid system which each mode is affine. The mode is determined by the control inputs and state (which determines the minimizer arguments). Some works consider nonlinear representations for the fundamental digram (Fig. 4), but they still can be approximated using piecewise affine functions.

VII-B Monotonicity

Theorem 5*.*

System (40) is monotone in $\Pi$.

Proof:

Consider $x^{\prime},x\in\Pi,x\preceq x^{\prime}$. We show that $f_{\text{traffic}}(x,u,w)\preceq f_{\text{traffic}}(x^{\prime},u,w),\forall w\in\mathcal{W},\forall u\in\mathcal{U}$. Observe in (39) that we only need to verify is proving that $x_{[l]}-\vec{q}_{[l]}$ is a non-decreasing function of $x_{[l]}$ as all other terms are additive and non-decreasing with respect to $x$. Since $x,x^{\prime}\in\Pi$, the last argument in (35) is never the minimizer. Thus, for all $\l\in\mathcal{L}$, we have $x_{[l]}-\vec{q}_{[l]}\in\{0,x_{[l]}-r_{[l]},x_{[l]}-c_{l},x_{[l]}\}$, depending on the mode of the system and actuations, which all are non-decreasing functions of $x_{[l]}$. Thus, $f_{\text{traffic}}$ is monotone in $\Pi$. ∎

The primary objective in our traffic management approach is finding control policies such that the state is restricted to $\Pi$, which not only eliminates congestion, but also ensures that the system is monotone hence the methods of this paper become applicable. It is worth to note that the traffic system becomes non-monotone when flow is congested in diverging junctions, as shown in [49]. This phenomena is attributed to the first-in-first-out (FIFO) nature of the model. By assuming fully non-FIFO models, system becomes monotone in the whole state space. For a more thorough discussion on physical aspects of monotonicity in traffic networks, see [15].

The maximal system in (40) corresponds to the scenario where each $w_{l}$ is equal to its maximum allowed value $w_{l}^{*}$.

VII-C Case Study

Network

Consider the network in Fig. 5, which consists of urban roads (links 1-26, 27,29,31,33 and 49-53), freeway segments (links 35-48) and freeway on-ramps (links 28,30,32,34). The layout of the network illustrates a freeway passing by an urban area, which is common in many realistic traffic layouts. There are 14 intersections (nodes a-n) controlled by traffic lights. Each intersection has two modes of actuation: north-south (NS) and east-west (EW). There are four entries to the freeway (nodes o-r) that are regulated by ramp meters. We have $n=53$ and $\mathcal{U}=\mathbb{R}_{+}^{4}\times\{0,1\}^{14}$. Vehicles arrive from links 1,6,11,15,19,23,35,42,49 and 52. The parameters of the network are shown in Table I.

Specification

As mentioned earlier, the primary objective is keeping the state in the congestion-free set. In addition, since the demand for the north-south side roads (links 49-53) is smaller than the traffic in the east-west roads, we add a timed liveness requirement for the traffic flow on links 49-53:

[TABLE]

which states that “if the number of vehicles on any of the north-south side roads exceeds 5, their flow is eventually actuated within three time units ahead”. The global specification is given as:

[TABLE]

Note that $h^{\varphi}=3$, $\varphi=(x\in\Pi)\wedge\psi$.

Open-loop Control Policy

We use Theorem 3. The shortest $\phi$-sequence that we found for this problem has $T=5,T_{0}=0$. The corresponding MILP had 2357 variables (of which 1061 were binary) and 4037 constraints 444The scripts for this case study are available in http://blogs.bu.edu/sadra/format-monotone, which is solved using the Gurobi MILP solver in less than 6 seconds on a dual core 3.0 GHz MacBook Pro. The cost is set to zero in order to just check for feasibility. Even though finding an optimal solution and checking for feasibility of a MILP have the same theoretical complexity, the latter is executed much faster in practice. For instance, finding a $\phi$-sequence, while minimizing or maximizing $\sum_{k=0}^{7}\|x^{\phi}_{k}\|_{1}$ both took more than 20 minutes. Note that it is virtually intractable to attack a problem of this size (53 dimensional state) using any method that involves state-space discretization, such as the method in [33] (e.g., if each state-component is partitioned into 2 intervals, the finite-state problem size will be $2^{53}$).

Monotonicity implies that any demand set $\mathcal{W}$ for which there exists a solution to Problem 2 is a lower-set. The set corresponding to the values at the bottom of Table I is one of them. Table II shows results on existence of $\phi$-sequences for some other demand scenarios. Computation times for solving a MILP do not demonstrate a generic behavior. For the rest of this section, the numerical examples are reported for the values in Table I.

The control values in the $\phi$-sequence are shown in Table III. As stated in Theorem 3, starting from an initial condition in $L(x_{0}$), applying the open-loop control policy (21) guarantees satisfaction of the specification. In other words, after applying the initialization segment, the repetitive controls in Table III become a fixed time-table for the inputs of the traffic lights and the ramp meters. Starting from $x_{0}$, which is a $53$-dimensional vector, we apply (21) using the values in Table III. The trajectory of the maximal system is shown in Fig. 6 [Top]. The traffic signals are coordinated such that the traffic flows free of congestion. The black dashed lines represent the capacity of the links, and the dashed line in the fourth figure (from the left) represents the threshold for the liveness sub-specification ($\psi$). It is observed that all the state values for side road links (49-53) persistently fall below the threshold. The robustness values for $(x\in\Pi)$ and $\psi$ are shown in the fifth figure. As mentioned earlier, robustness corresponds to the minimum volume of vehicles that the system is away from congestion, or violating the specification. The robustness values are always positive, indicating satisfaction.

As stated in Theorem 4, the trajectory of the maximal system converges to a periodic orbit. It is worth to note that the number of vehicles on freeway links is significantly smaller than its capacity, which is attributed to the fact that the number designated for $\bar{q}$ (related to the maximum speed) of freeway links is relatively large (30, as opposed to 15 for roads). Therefore, freeway links are utilized in a way that there is enough space for high speed non-congested flow.

Robust MPC

Here it is assumed that the controller has full state knowledge. We apply the techniques developed in Sec. VI. Using the result from the previous section, the set $\Omega_{\mathcal{L}^{\varphi}}$ is constructed in $\mathbb{R}_{+}^{212}$ ($=\mathbb{R}^{n(h^{\varphi}+1)},n=53,h^{\varphi}=3$). The cost criteria that we use in this case study is the total delay induced in the network over the planning horizon $H$. A vehicle is delayed by one time unit if it can not flow out of a link in one time step, which may be because of the actuation (e.g., red light) or waiting for the flow of other vehicles in the same link (i.e., we have $x_{[l]}\geq c_{l}$). We are also interested in maximizing the STL robustness score. The cost function is:

[TABLE]

where $\vec{q}_{[l]}$, given by (38), is the amount of vehicles that flow out of link $l$, $\gamma$ is the discount factor for delays predicted in further future, and $\zeta$ is a positive weight for robustness. Notice the connection between the time window of STL robustness score in (42) and MPC constraint enforcement in (33). It follows from Theorem 5 and STL quantitative semantics (4) that the cost function above is non-decreasing with respect to the state in $\Pi$. Therefore, in order to minimize the worst case cost, the maximal system is considered in the MPC optimization problem.

Starting from zero initial conditions, we implement the MPC algorithm (34) with $H=3$ for 40 time steps. We set $\zeta=1000$, $\gamma=0.5$ in (42). The disturbances at each time step were randomly drawn from $L(w^{*})$ using a uniform distribution. The maximum computation time for each MPC step time step was less than 0.8 seconds (less than 0.5 seconds on average). The resulting trajectory is shown in Fig. 6 [Middle]. For the same sequence of disturbances, the trajectory resulted from applying the open-loop control policy (21) (using the values in Table III) is shown in Fig. 6 [Bottom]. Both trajectories satisfy the specification. However, robust MPC has obviously better performance when costs are considered. The total delay accumulated over 40 time steps is:

[TABLE]

The cost above obtained from applying robust MPC was $J_{40}=1843$, while the one for the open-loop control policy was $J_{40}=2299$, which demonstrates the usefulness of the state knowledge in planning controls in a more optimal way. An optimal tuning of parameters $\eta$ and $\gamma$ requires an experimental study which is out of scope of this paper. We only remark that we usually obtained larger delays with non-zero $\eta$, which shows that including STL robustness score in the MPC cost function may be useful even though the ultimate goal is minimizing the total delay.

It is worth to note that we also tried implementing the MPC algorithm (for the case ${\bf{{w}}}=(w^{*})^{\omega}$, or the maximal system) without the terminal constraints, as in (32). The MPC got infeasible at $t=8$. The violating constraints were those in $x\in\Pi$. This observation indicates that the myopic behavior of MPC in (32), when no additional constraints are considered, can lead to congestion in the network.

VIII Conclusion and Future Work

We developed methods to control positive monotone discrete-time systems from STL specifications. We showed that open-loop control sequences are sufficient and (almost) necessary for guaranteeing the correctness of STL specifications. A robust MPC method was introduced to plan controls optimally, while guaranteeing global STL specifications. We showed the usefulness of our results on traffic management.

Future work will focus on non-monotone systems with parametric uncertainty whose state evolution can be over-approximated in an appropriate way using monotone systems. We will develop adaptive control schemes to tune parameters automatically using the data gathered from the evolution of the system. This will eventually lead to data-driven control techniques for transportation networks with formal guarantees.

Appendix

Theorem 6*.*

Let $\mathbb{S}$ be the set of all STL formulas that can be written in the form:

[TABLE]

where $\varphi_{b,i},\Delta_{i}\geq h^{\varphi_{b,i}},\varphi_{g,i},i=1,\cdots,n_{\phi},$ are bounded STL formulas. Then $\mathbb{S}$ is a subset of safety STL formulas that is closed under STL syntax with bounded temporal operators.

Proof:

First, a quick inspection of (43) verifies that it is a safety STL formula. A predicate $\pi$ is a bounded formula (with zero horizon) and is a special case of (43), hence $\pi\in\mathbb{S}$.

We also have the following property that relaxes the form in (43): For all bounded STL formulas $\varphi_{1},\varphi_{2}$, we have $\varphi_{1}\wedge{\bf{{G}}}_{[\Gamma,\infty)}\varphi_{2}\in\mathbb{S}$, $\forall\Gamma\in\mathbb{N}$. Proof: The case for $\Gamma\geq h^{\varphi_{1}}$ is already in the form (43) with $n_{\phi}=1$. If $\Gamma<h^{\varphi_{1}}$, we write $\mathbb{G}_{[\Gamma,\infty)}\varphi_{2}=\mathbb{G}_{[\Gamma,h^{\varphi_{1}}]}\varphi_{2}\wedge\mathbb{G}_{[h^{\varphi_{1}},\infty)}\varphi_{2}$. Now, define $\varphi_{1}\wedge\mathbb{G}_{[\Gamma,h^{\varphi_{1}}]}\varphi_{2}$ as the new bounded formula and retain the form in (43) with $n_{\phi}=1$.

We show that $\mathbb{S}$ is closed under STL syntax with bounded operators. The distributivity properties of Boolean connectives and temporal operators (see, e.g., [50]) imply that: $\phi_{1}\vee(\phi_{2}\wedge\phi_{3})=(\phi_{1}\vee\phi_{2})\wedge(\phi_{2}\vee\phi_{3})$, $\phi_{1}\wedge(\phi_{2}\vee\phi_{3})=(\phi_{1}\wedge\phi_{2})\vee(\phi_{2}\wedge\phi_{3})$, ${\bf{{F}}}_{I}(\phi_{1}\vee\phi_{2})=({\bf{{F}}}_{I}\phi_{1})\vee({\bf{{F}}}_{I}\phi_{2})$, and ${\bf{{G}}}_{I}(\phi_{1}\wedge\phi_{2})=({\bf{{G}}}_{I}\phi_{1})\wedge({\bf{{G}}}_{I}\phi_{2})$, where $\phi_{1},\phi_{2},\phi_{3}$ are temporal logic formulas and $I$ is an interval.

$\phi_{1},\phi_{2}\in\mathbb{S}\Rightarrow\phi_{1}\wedge\phi_{2}\in\mathbb{S},\phi_{1}\vee\phi_{2}\in\mathbb{S}$: this result easily follows from the distributivity properties of Boolean connectives mentioned above. 2. 2.

$\phi\in\mathbb{S}\Rightarrow{\bf{{F}}}_{\{t\}}\phi\in\mathbb{S}$: we use ${\bf{{F}}}_{\{t\}}{\bf{{G}}}_{[a,b]}={\bf{{G}}}_{[t+a,t+b]}$ and distributivity to have (note that ${\bf{{F}}}_{\{t\}}={\bf{{G}}}_{\{t\}}$)

[TABLE]

Introducing ${\bf{{F}}}_{\{t\}}\varphi_{b,i},i=1,\cdots,n_{\phi}$, as new bounded STL formulas leads to the form in (43). 3. 3.

$\phi\in\mathbb{S}\Rightarrow{\bf{{F}}}_{[a,b]}\phi\in\mathbb{S},{\bf{{G}}}_{[a,b]}\phi\in\mathbb{F}$: use ${\bf{{F}}}_{[a,b]}=\bigvee_{t\in[a,b]}{\bf{{F}}}_{\{t\}}$ and ${\bf{{G}}}_{[a,b]}=\bigwedge_{t\in[a,b]}{\bf{{F}}}_{\{t\}}$ to convert temporal operators to Boolean connectives. 4. 4.

$\phi_{1},\phi_{2}\in\mathbb{S}\Rightarrow\phi_{1}{\bf{{U}}}_{[a,b]}\phi_{2}\in\mathbb{S}$: use the STL semantics (2) to substitute the bounded “until” operator using bounded “eventually” and bounded “always” operators:

[TABLE]

Example 6*.*

The “reach and stay” formula ${\bf{{F}}}_{I}{\bf{{G}}}_{[0,\infty)}\varphi$, where $\varphi$ is a bounded formula, is equivalent to $\bigvee_{t\in I}{\bf{{G}}}_{[t,\infty)}\varphi$.

Remark 5*.*

What remains to show that $\mathbb{S}$ is equivalent to the set of all safety STL formulas is having that \phi\in\mathbb{S}\Rightarrow{\bf{{G}}}_{[{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}{\Gamma}},\infty)}\phi\in\mathbb{S},\forall\Gamma\in\mathbb{N}, which is not true by restricting $n_{\phi}$ in (43) to be finite. Formulas that involve nested unbounded “always” operator and can not be further simplified, such as ${\bf{{G}}}_{[\Gamma^{\prime},\infty)}(\varphi_{1}\vee{\bf{{G}}}_{[\Gamma,\infty)}\varphi_{2})$, are rarely encountered in applications.

∎