Integrated Offline and Online Optimization-Based Control in a Base-Parallel Architecture

TL;DR

This paper introduces an integrated control architecture combining offline-tuned base controllers with online optimization-based parallel controllers, enabling efficient real-time control of nonlinear systems with constraints, demonstrated on highway traffic management.

Contribution

The paper presents a novel flexible control architecture that integrates offline and online controllers for real-time nonlinear system management, adaptable online for performance improvements.

Findings

Achieves real-time control within strict time budgets.

Outperforms traditional online MPC in computational efficiency.

Reduces overall system cost in traffic management case study.

Abstract

We propose an integrated control architecture to address the gap that currently exists for efficient real-time implementation of MPC-based control approaches for highly nonlinear systems with fast dynamics and a large number of control constraints. The proposed architecture contains two types of controllers: base controllers that are tuned or optimized offline, and parallel controllers that solve an optimization-based control problem online. The control inputs computed by the base controllers provide starting points for the optimization problem of the parallel controllers, which operate in parallel within a limited time budget that does not exceed the control sampling time. The resulting control system is very flexible and its architecture can easily be modified or changed online, e.g., by adding or eliminating controllers, for online improvement of the performance of the controlled system. In a case study, the proposed control architecture is implemented for highway traffic, which is characterized by nonlinear, fast dynamics with multiple control constraints, to minimize the overall travel time of the vehicles, while increasing their total traveled distance within the fixed simulation time window. The results of the simulation show the excellent real-time (i.e., within the given time budget) performance of the proposed control architecture, with the least realized value of the overall cost function. Moreover, among the online control approaches considered for the case study, the average cost per vehicle for the base-parallel control approach is the closest to the online MPC-based controllers, which have excellent performance but may involve computation times that exceed the given time budget.

Tables

Table 1. Table I: Simulated values of the parameters used for the ACTM and for the control system in the case study.

parameter name	simulation and control sampling cycle	cell length	free-flow speed	idling speed	average vehicle length555The given value for the average vehicle length is in line with the findings of a project performed in 2015 by the New England Section of the Institute of Transportation Engineers Technical Committee, which is available via http://neite.org/Documents/Technical/.	cell capacity
simulated value	$20$ [s]	$560$ [m]	$28$ [m/s]	$6.5$ [m/s]	$7$ [m]	$80$ [veh]

Table 2. Table II: The results of a 1-hour simulation for the case study.

control approach	$J^{\text{total}}=\sum _k\left(\text{TT}\left(k\right)-\gamma \text{TD}\left(k\right)\right)$ [h]	$n^{\text{total}}$ [veh]	average cost per vehicle [s]
Alinea	455.95	6.1342 $\times {10}^4$	26.76
ANN	431.71	6.4531 $\times {10}^4$	24.08
CMPC(1)	412.76	6.7738 $\times {10}^4$	21.94
CMPC(2)	412.76	6.7738 $\times {10}^4$	21.94
PMPC(1)	412.91	6.7710 $\times {10}^4$	21.95
PMPC(2)	412.91	6.7679 $\times {10}^4$	21.96
Base-parallel architecture	412.69	6.7582 $\times {10}^4$	21.98

Equations

$$\mathbf{u}(k)=\hat{f}\left(k,{\mathbf{x}}^{\text{m}}(k),{\mathbf{\nu }}^{\text{m}}(k)\right).$$

$$\mathbf{u}(k)=\hat{u}\left(\mathbf{\theta }(k),k,{\mathbf{x}}^{\text{m}}(k),{\mathbf{\nu }}^{\text{m}}(k)\right),$$

$$b(k,{\mathbf{x}}^{\text{m}}(k))=\underset{\tilde{\mathbf{u}}(k;n^{\text{H}})}{\min }\left\{\sum _{\kappa =k}^{k+n^{\text{H}}-1}\hat{c}(\mathbf{x}(\kappa ),\mathbf{u}(\kappa ))+{\hat{c}}^{\text{t}}(\mathbf{x}(k+n^{\text{H}}))\right\},$$

$$n_i(k^{\text{s}}+1)$$

$$q_i(k^{\text{s}}+1)$$

$$\displaystyle o_{i}(k^{\textrm{s}})=\min\bigg{\{}$$

$$\left({\bar{n}}_{i+1}-n_{i+1}(k^{\text{s}})-{\alpha }_{i+1}(k^{\text{s}})e_{i+1}(k^{\text{s}})\right){\eta }_{i+1}^{\text{i}}(k^{\text{s}}),$$

$$\displaystyle\bar{o}_{i},\frac{\displaystyle 1-\beta_{i}(k^{\textrm{s}})}{\beta_{i}(k^{\textrm{s}})}\bar{s}_{i}\bigg{\}}.$$

$$\displaystyle e_{i}(k^{\textrm{s}})=\left\{\begin{array}[]{l}\min\bigg{\{}q_{i}(k^{\textrm{s}})+d_{i}(k^{\textrm{s}}),\xi_{i}\left(\bar{n}_{i}-n_{i}(k^{\textrm{s}})\right)\bigg{\}},\\ \textrm{\hskip 51.6665pt if the on-ramp is not metered}\\ \\ \min\bigg{\{}q_{i}(k^{\textrm{s}})+d_{i}(k^{\textrm{s}}),\xi_{i}\left(\bar{n}_{i}-n_{i}(k^{\textrm{s}}),\mu_{i}(k^{\textrm{s}})\right)\bigg{\}},\\ \textrm{\hskip 51.6665pt if the on-ramp is metered}\end{array},\right.$$

$$s_i(k^{\text{s}})=\frac{{\beta }_i(k^{\text{s}})}{1-{\beta }_i(k^{\text{s}})}{o}_i(k^{\text{s}}).$$

$$\mu_{i}(k)=\max\Big{\{}\mu_{i}(k-1)+\theta^{\textrm{ALINEA}}_{i}(k)\left(\rho^{\textrm{crit}}_{i}-\rho_{i}(k)\right),0\Big{\}},$$

$$n_i^{\text{exp}}(k+1)=n_i(k)+o_{i-1}(k)+e_i(k)-o_i(k)-s_i(k),$$

$$J(k)=\text{TT}(k)-\gamma \text{TD}(k),$$

Full text

I Introduction: objective, contributions, and structure

In this paper, we propose and develop a novel control architecture that integrates several control approaches in a smart and efficient way to create a well-performing, real-time MPC-based control approach for nonlinear systems with fast dynamics and a large number of control constraints. Our key objective is to close the existing gap in real-time optimization-based control of such systems, which appear frequently in real-life applications and demand fast accurate control approaches.

The main contributions of this paper include:

• •

We propose an integrated control system comprising multiple offline tuned or optimized and online optimization-based controllers within a novel base-parallel architecture. This architecture first determines several candidate control inputs for the controlled system at every control sampling time step, and next selects the one resulting in the optimal performance.

• •

Our main achievement via the proposed control architecture is efficient real-time (i.e., within a time budget that does not exceed the control sampling time) optimal control of nonlinear systems with fast dynamics and many control constraints, which is currently not tractable in real time.

• •

A special aspect of the proposed control architecture is its flexibility for embedding and incorporating various control policies. Moreover, the architecture represents a partially-interconnected multi-agent control system with the capacity of adding or eliminating agents without changing the existing structure.

The next section summarizes the main characteristics of offline tuned or optimized and online optimization-based controllers, and the existing control gaps, which cannot be filled with either of these control strategies alone, but through the integrated architecture we propose in this paper. In Section III, we explain the proposed control architecture in detail. Section IV includes a case study, Section V gives a general discussion of the results, and Section VI concludes the paper and discusses some remaining aspects of this topic that should further be investigated in the future.

II Control categorization

Controllers may be categorized in various ways, depending on the application of interest. In this paper, we divide controllers into two general categories, offline and online, based on how the controller is tuned or optimized.

A control law can, in general, be formulated as a mathematical expression that relates the control input $\bm{u}(k)$ with the control sampling time step $k$, the measured state variable $\bm{x}(k)$, and the measured uncontrolled external input $\bm{\nu}^{\textrm{m}}(k)$, i.e.,

[TABLE]

The function $\hat{f}(\cdot)$ may or may not vary while the controller runs (e.g., $\hat{f}(\cdot)$ may change from a polynomial to a trigonometric expression). Hence, an equivalent mathematical expression for (1) is a parameterized formulation:

[TABLE]

where $\hat{u}(\cdot)$ has a fixed mathematical expression, but $\bm{\theta}$, i.e., the vector of the control input parameters, may vary.

Remark 1

The discrete-time domain is used throughout this paper, with $k$ the discrete sampling time step counter. Bold and regular letters are used for, respectively, vectors and scalars. For functions, a hat symbol is used on a small regular letter. The superscripts “m” and “p” indicate “measured” and “predicted”.

II-A Offline tuned or optimized control

In case the parameter vector $\bm{\theta}$ in (2) is fixed, or is tuned or optimized offline (i.e., parallel to running the control procedure), we call such control policies offline tuned or optimized control strategies. To tune $\bm{\theta}$ offline, an extensive training dataset is used that includes pairs of state variables that cover the system’s entire state space and control inputs that result in a desired behavior for the controlled system (see Figure 2). These pairs are selected in either of the following ways: from previous control sampling time steps, when the realized behavior of the controlled system has resulted in certain criteria to be satisfied, for instance when the performance is above a desired threshold or the CPU time has been exhausted; from future control sampling time steps, when the estimated behavior of the controlled system, using a mathematical model for predicting the state variables and an offline optimizer for determining the control inputs, can result in certain criteria to be satisfied.

II-B Online optimization-based control

In online optimization-based control, the parameter vector $\bm{\theta}$ in (2) is optimized online at every control sampling time step, and the resulting optimal $\bm{\theta}$ is used in (2) to evaluate the control input $\bm{u}(k)$ for that control sampling time step. The optimization is performed in a prediction time window of length $n^{\textrm{H}}$ rather than at a single control sampling time step. Therefore, at every control sampling time step $k$, a sequence $\tilde{\bm{\theta}}(k;n^{\textrm{H}})$ of optimal parameter vectors, or a sequence $\tilde{\bm{u}}(k;n^{\textrm{H}})$ of the corresponding control inputs are determined, computing the Bellman value function $b(k,\bm{x}^{\textrm{m}}(k))$ via minimizing the summation of the cumulative value of a performance index function $\hat{c}(\cdot,\cdot)$ within the prediction time window and a terminal cost value $\hat{c}^{\textrm{t}}\left(\bm{x}(k+n^{\textrm{H}})\right)$, i.e.,

[TABLE]

subject to the physical and control constraints of the problem.

Since the target of this paper is to develop an optimization-based control architecture, we next discuss some existing optimization-based control methods and explain to which of the two categories introduced above they belong.

II-C Existing optimization-based controllers

In open-loop optimal control (top plot in Figure 2), the optimal control input sequence is determined offline (before the control procedure starts) and the online optimization, which usually demands more time than the operational sampling time, is eliminated. Open-loop optimal control, hence, belongs to the category of offline optimized control. Since the state variables measured for solving the offline optimization problem may evolve considerably while the offline computations are running, the accuracy and reliability of the resulting control input sequence is under question. Moreover, this sequence is determined assuming perfect prediction models for the state variable and the external input, which in reality is not true.

In closed-loop optimal control, i.e., conventional model-predictive control (MPC) [1], the optimal111Note that since the length of the MPC optimization time window is usually smaller than the length of the control time window, the resulting control input sequence is, in general, suboptimal for the closed-loop performance. control input sequence is determined online at every control sampling time step based on the most recent measurements of the state variable and the external inputs (bottom plot in Figure 2). The first element of the optimal control input sequence is injected into the controlled system and the prediction time window is shifted forward to the next control sampling time step. Classically, MPC is treated as an online optimization-based control policy, i.e., the optimal control input sequence is determined solving the optimization problem of MPC online (see, e.g., [2, 3]). Online MPC should solve the optimization problem within one control sampling cycle, which is determined based on the speed of the dynamics of the controlled system and external inputs. This had traditionally restricted the applicability of online MPC to systems with slow dynamics (of order minute or second [4]). More recent work, e.g., [4, 5], have made online linear MPC and online RTI-based nonlinear MPC applicable to faster dynamics (of order millisecond and microsecond). To tackle the computational complexities of online optimization, some approaches have been developed, which treat MPC as an offline optimized control strategy. Next, we discuss the following types of MPC: multi-parametric or explicit MPC [6, 7, 8, 9], combined multi-parametric and online MPC [10], parameterized [11, 12] and move blocking [13, 14] MPC, and nonlinear MPC (NMPC) [15].

Multi-parametric MPC formulates the MPC control inputs as a set of explicit functions of the system’s state variables. Each function is an offline solution of the optimization problem in a particular subregion of the system’s state space, treating the initial state as a parameter. The online computations are reduced to the evaluation of the corresponding function for the measured state. Hence, multi-parametric MPC lies within the category of offline optimized control. In general, the number of the state space subregions grows exponentially with the number of the state components and constraints, which implies a large CPU time for the computations and huge memory requirements for storing the parametric solutions. Bemporad and Filippi [16] propose approximate solutions (by relaxing some of the first-order KKT optimality conditions) for explicit MPC with a reduced number of state space subregions, which despite the positive influence on the CPU time and required memory, may however negatively affect the optimality.

The next MPC strategies belong to the online optimization-based control category. In combined multi-parametric and online MPC [10], the control input sequence evaluated by multi-parametric MPC is used at every control sampling time step as a starting optimization point for the online MPC. This method has been applied to quadratic programming in [10].

In contrast to conventional MPC (see the top left plot in Figure 3), where $\hat{f}\left(\cdot\right)$ in (1) or $\bm{\theta}$ in (2) are optimized for every control sampling time step in the prediction time window, in parameterized MPC and move blocking MPC, which lie within the category of online MPC, $\hat{f}\left(\cdot\right)$ or $\bm{\theta}$ are optimized subject to an extra constraint that reduces the degree of freedom. In parameterized MPC, $\bm{\theta}$ is fixed for all control sampling time steps within the prediction time window, and the number of the elements of the parameter vector $\bm{\theta}$ is considered less than the number of elements of the original control input sequence (see the top right plot in Figure 3). In move blocking MPC, $\hat{f}\left(\cdot\right)$ or $\bm{\theta}$ follow a blocking stepwise function, i.e., the optimization variables are constant for multiple consecutive control sampling time steps (see the bottom plots in Figure 3). In summary, in parameterized and move blocking MPC the number of the optimization variables compared to conventional MPC is reduced, so that the computation time may become smaller. A main drawback is that since the solutions are limited to the family of the introduced parametric function, only a subregion of the feasible optimization region is searched, while the optimal value(s) of the original optimization problem may or may not belong to this subregion.

NMPC is preferred over linear MPC when the nonlinear dynamics and constraints are better treated explicitly (without linearization) due to the accuracy requirements. Initially, online NMPC was used in the process industry for nonlinear systems with slow dynamics [15, 17]. Nowadays, there are fast NMPC approaches, such as the quadratic programming (SQP) and real-time iteration (RTI) that can operate online for control sampling times in the range of millisecond [5, 18, 19].

II-D Existing gaps in online MPC-based control

Figure 4 illustrates, in one glance, the fields of online applicability of the discussed MPC-based approaches. For nonlinear systems involving fast dynamics and a large number of constraints, a gap exists that cannot yet be addressed via the existing MPC-based control approaches. Despite the good performance of NMPC and RTI-based MPC with dynamics as fast as millisecond and microsecond [18, 5], as the complexity level, e.g., nonlinearity, and the number of constraints rise, the applicability of these approaches becomes more restricted.

The theory and application of online MPC for nonlinear systems needs to be further developed. The main aim of this paper is to take significant steps towards closing this gap by proposing a novel control architecture that integrates several control approaches in a smart and efficient way to create a well-performing, real-time222The definition of “real-time” in this paper is relative, i.e., it depends on the dynamics evolution of the system and correspondingly, the control sampling time. A control approach is real-time for a specific controlled system, if the computation time of the control approach does not exceed the control sampling time of the controlled system. MPC-based control approach for nonlinear systems with a large number of constraints.

III Base-Parallel Integrated Control Architecture

In this section, the proposed integrated control architecture is explained in detail. Figure 5 shows the main components of the control architecture, i.e., the base block and the parallel block. The base block consists of multiple controllers that are tuned or optimized offline, called base controllers. The parallel block includes several online optimization-based controllers, called parallel controllers, which run in parallel during the online control procedure. The control architecture, also including an evaluation block composed of identical mathematical models of the controlled system and a selector, is feedback-based. Some measurements, e.g., the realized state variables, the uncontrollable external inputs, and the performance indices, from the controlled system are fed back into the control architecture. The different control modules, which have been illustrated in detail in Figure 6, will next be explained.

III-A Base control block

Considering the structure of the control policy, a base controller may be fixed-time, i.e., the function $\hat{f}(\cdot)$ in (1) or equivalently the parameter vector $\bm{\theta}$ in (2) are fixed in time. Therefore, the same mathematical formulation is used at all control sampling time steps to evaluate the control input $\bm{u}$ from the measured values of the state variable and uncontrolled external inputs. Control policies such as open-loop optimal control, multi-parametric MPC, full-state feedback control (pole placement), offline optimized state feedback control, and PID control may be optimized or tuned offline and used as fixed-time base controllers.

Some base controllers may be time-varying, i.e., the function $\hat{f}(\cdot)$ in (1) or equivalently the parameter vector $\bm{\theta}$ in (2) can vary in time. A mapping, e.g., an artificial neural network (ANN), that has been formulated and trained offline using an extensive dataset collected via performing simulations or from the past desired behavior of the controlled system, is used to evaluate $\bm{\theta}$ or equivalently the structure of $\hat{f}(\cdot)$. This evaluation may be done at every control sampling time step or less frequently. Note that since the trained mapping involves algebraic computations, the online computation time is negligible. This is in contrast to an optimizer that involves an online optimization procedure with recursive differential computations and estimations, and hence may suffer from online computational complexities (see Figure 7).

Considering the structure of the output of a base controller, the following two categories may exist:

III-A1 Explicit base controller

Such a controller produces the control input $\bm{u}$ directly at every control sampling time step.

III-A2 Implicit base controller

Such a controller produces the parameter vector $\bm{\theta}$ in (2) at every control sampling time step.

III-B Parallel control block

Considering the structure of the output, a parallel controller may belong to either of the following two categories:

III-B1 Explicit parallel controller

At every control sampling time step $k$, such a controller directly optimizes the sequence $\tilde{\bm{u}}\left(k;n^{\textrm{H}}\right)$ of the control inputs for all the control sampling time steps within the prediction time window. A conventional MPC-based controller can, e.g., be used as an explicit parallel controller.

III-B2 Implicit parallel controller

At every control sampling time step $k$, such a controller uses the formulation (2) to optimize the sequence $\tilde{\bm{\theta}}(k;n^{\textrm{H}})$ of the control parameter vectors within the prediction time window. A parameterized MPC-based controller is an example of an implicit parallel controller.

A CPU time budget is given to the parallel block for solving the optimization problems. When this time budget is exhausted, all the optimization procedures will be terminated. If the optimization problem has been solved within the given time budget, the optimal solution will be the candidate control input sequence of the corresponding parallel controller. Otherwise, if an optimization procedure has been terminated before an optimal solution that satisfies the optimization criteria was found, two options are possible:

The realized values of the cost for all optimization iterations are saved together with their corresponding control input sequences. After the termination of the optimization, these values are compared and the control input sequence that corresponds to the least realized optimization cost is selected as the candidate control input of that parallel controller. 2. 2.

All control input sequences that were determined at the optimization iterations are injected into the evaluation block as candidate control input sequences of that parallel controller. Note that this option can be beneficial when the mathematical model used in the evaluation block (see Section III-D for more details) is different from (i.e., contains more details than) the prediction model of the parallel controller, and the computational burden for evaluating all the cost values corresponding to these candidate control input sequences by the evaluation block is affordable for the control system.

III-C Integrated structure of the base and parallel blocks

Figure 6 illustrates the proposed integrated control architecture in more detail. At every control sampling time step $k$, the measured values of the state variable, $\bm{x}^{\textrm{m}}(k)$, and uncontrolled external input, $\bm{\nu}^{\textrm{m}}(k)$, are injected into the base, parallel, and evaluation blocks. The implicit and explicit base controllers, indicated by BCi and BCe in the figure, will evaluate the control parameter vector $\bm{\theta}^{\textrm{BCi}}(k)$ and the control input $\bm{u}^{\textrm{BCe}}(k)$ respectively, using these measurements. The vectors $\bm{\theta}^{\textrm{BCi}}(k)$ are used to evaluate the corresponding control inputs $\bm{u}^{\textrm{BCi}}(k)$ using (2). The vector $\bm{u}^{\textrm{BC}}(k)$, including all candidate control inputs $\bm{u}^{\textrm{BCe}}(k)$ and $\bm{u}^{\textrm{BCi}}(k)$ from the explicit and implicit base controllers, is injected into the evaluation block.

Moreover, in order for the parallel controllers to start from a starting optimization point with a higher chance of converging to an optimum within the given CPU time budget, the control inputs corresponding to the base controllers are injected into the parallel block as starting points for the optimization. This is a key aspect of the proposed integrated control architecture. Before we explain the details of how this idea is implemented in the integrated base-parallel architecture, the terminology that will be used is explained. In Figure 6, every row of parallel controllers that is indicated by a dashed rectangle is called a parallel cell. The elements of the optimization variable sequences of the parallel controllers in one parallel cell are of the same nature, i.e., either explicit control input vectors or control input parameter vectors. However, the number of the elements in the sequences, i.e., the control horizon of the parallel controllers in one cell, may be different from each other. Therefore, a parallel cell is either an explicit parallel cell, i.e., all parallel controllers in the cell are explicit, or is an implicit parallel cell, i.e., all parallel controllers in the cell are implicit. Each parallel cell corresponds to a particular base controller, i.e., the parallel controllers in that parallel cell receive the starting optimization sequence computed by this particular base controller. Hence, an explicit parallel cell corresponds to an explicit base controller, and an implicit parallel cell corresponds to an implicit base controller.

Since the parallel controllers follow model-based and optimization-based control strategies that compute the control input or the control parameter vector within a prediction time window (including the current and the $n^{\textrm{H}}-1$ future control sampling time steps), these controllers optimize a sequence of either the control input or the control parameter vector. Therefore, the parallel controllers should initially receive a sequence of the control input or control parameter vector to start the optimization procedure. The base controllers, however, produce the control input or control parameter vector for the current control sampling time step only. Hence, the control input or the control parameter vector should first be estimated for the control sampling time steps $k+1,\ldots,k+n^{\textrm{H}}-1$ based on the one computed by the base controller for the current control sampling time step (see Figure 8).

A mathematical model of the states of the controlled system, integrated with a mathematical model of the uncontrolled external inputs is used for this estimation. This model should perform in a loop (of size the largest prediction horizon of the corresponding parallel controllers) together with the base controller: for the control sampling time steps $\kappa\in\left\{k,\ldots,k+n^{\textrm{H}}-1\right\}$, the control input at the control sampling time step $\kappa$ computed by the base controller is sent to the integrated mathematical model, which estimates the states of the controlled system based on the received control input, and the uncontrolled external inputs for the next control sampling time step $\kappa+1$. These estimated values are sent back to the base controller to compute the control input at $\kappa+1$. This loop has been illustrated in Figure 6. The size of the loop is determined via the size of the prediction horizon of the corresponding parallel controllers. Therefore, the maximum size of all the prediction horizons in the parallel cell (see Figure 6) is given to the loop, such that the loop generates a sequence of control inputs of this maximum size. Every parallel controller in the parallel cell receives as the starting point for the optimization the first element of this sequence to the element of the number of its prediction horizon. Note that for an implicit base controller, the control input is first evaluated via (2) using the control parameter vector produced by the implicit base controller.

III-D Evaluation block and selector

At every control sampling time step $k$, all the control inputs determined by the base and parallel controllers (indicated by $\bm{u}^{\textrm{BC}}(k)$ and $\bm{u}^{\textrm{PC}}(k)$ in Figure 6) enter the evaluation block. This block consists of identical integrated mathematical models, which estimate the states of the controlled system based on the received control input, and also estimate the uncontrolled external inputs. Note that since the evaluation block has to run only once for every control sampling time step, these integrated models can be much more refined and detailed than the models used for the base and parallel controllers. The number of these models, in general, equals the total number of the base and parallel controllers in the integrated control architecture. These models will run in parallel to evaluate a predefined performance index function $\hat{\epsilon}(\cdot)$ for the controlled system for each received candidate control input. The resulting values (indicated by vector $\bm{\epsilon}(k)$ in Figure 6) are sent to a selector that compares them and selects the control input corresponding to the least realized value of the performance index as the best candidate $\bm{u}^{\textrm{best}}(k)$ at control sampling time step $k$ to be injected into the controlled system.

IV Case Study

In this section, we present the results of a case study, where the proposed integrated base-parallel control architecture is implemented to a highway stretch to increase the total distance traveled by the vehicles during the fixed simulation time window, while reducing their total travel time.

IV-A Highway simulation

The model we use in this paper as simulation model and as prediction models of the MPC-based controllers is the asymmetric cell transmission model (ACTM)333Note that the approach proposed in this paper is generic, and other models could easily be used instead of the ACTM. [20]. Figure 9 illustrates via a stretch of a road with an on-ramp and an off-ramp the main concepts and variables used by the ACTM. The stretch of the road is divided into sections, called cells, where each cell includes maximally one on-ramp and maximally one off-ramp. When a cell contains both an on-ramp and an off-ramp, the on-ramp should be upstream of the off-ramp. Each cell is identified by an index $i$, where an on-ramp and an off-ramp that belong to a cell adopt the same index as the cell.

The main variables of the ACTM are:

• •

$k^{\textrm{s}}$: simulation sampling time step counter

• •

$n_{i}(k^{\textrm{s}})$: total number of vehicles in cell $i$ for the simulation sampling time step $k^{\textrm{s}}$

• •

$q_{i}(k^{\textrm{s}})$: total number of vehicles that are queuing on the on-ramp $i$ for the simulation sampling time step $k^{\textrm{s}}$

• •

$o_{i}(k^{\textrm{s}})$: mainline outflow of cell $i$/mainline inflow of cell $i+1$ (i.e., total number of vehicles that leave cell $i$ towards cell $i+1$) for the simulation sampling time step $k^{\textrm{s}}$

• •

$s_{i}(k^{\textrm{s}})$: off-ramp outflow of cell $i$, i.e, total number of vehicles that leave cell $i$ via off-ramp $i$ for the simulation sampling time step $k^{\textrm{s}}$

• •

$e_{i}(k^{\textrm{s}})$: on-ramp inflow of cell $i$ (or outflow of on-ramp $i$), i.e., total number of vehicles that enter cell $i$ via on-ramp $i$, for the simulation sampling time step $k^{\textrm{s}}$

• •

$\alpha_{i}(k^{\textrm{s}})$: percentage of the vehicles that enter cell $i$ via on-ramp $i$ for the simulation sampling time step $k^{\textrm{s}}$ and can blend with the existing moving flow in the cell, i.e., the ACTM assumes two traffic regimes for the vehicles that enter a cell via an on-ramp; moving and idling, and correspondingly defines a blending coefficient. The vehicles that belong to the idling regime, will stay in the cell during the current simulation sampling cycle, while all or part of the vehicles in the moving regime may leave the cell during the current simulation sampling cycle. The vehicles that are already in the cell at the beginning of the current simulation sampling cycle, belong to the moving regime. Note that in the limit, when the cell is congested, the moving and the idling regimes will merge and adopt the same speed.

• •

$d_{i}(k^{\textrm{s}})$: demand, given in the number of vehicles per simulation sampling cycle, at the beginning of on-ramp $i$ for the simulation sampling time step $k^{\textrm{s}}$

• •

$\bar{o}_{i}$: mainline saturation outflow of cell $i$, i.e., maximum number of vehicles that can leave cell $i$ towards cell $i+1$ within one simulation sampling cycle

• •

$\bar{s}_{i}$: off-ramp saturation outflow of cell $i$, i.e., maximum number of vehicles that can leave cell $i$ via off-ramp $i$ within one simulation sampling cycle

• •

$\bar{n}_{i}$: capacity of cell $i$, i.e., maximum number of vehicles that can be in cell $i$

• •

$\beta_{i}(k^{\textrm{s}})$: percentage of the vehicles that leave cell $i$ for the simulation time step $k^{\textrm{s}}$ via off-ramp $i$

• •

$\eta^{\textrm{m}}_{i}(k^{\textrm{s}})$: percentage of the vehicles in the moving regime of cell $i$ for the simulation sampling time step $k^{\textrm{s}}$ that can leave this cell within the current simulation sampling cycle, where this percentage depends on the moving speed the vehicles adopt in the cell for this simulation sampling time step. In the limit, when the cell and its neighboring consecutive cells are congested, $\eta^{\textrm{m}}_{i}(k^{\textrm{s}})\hskip 0.6458pt\rightarrow\hskip 0.6458pt0$

• •

$\eta^{\textrm{i}}_{i}(k^{\textrm{s}})$: percentage of the vacant capacity in cell $i$ for the simulation sampling time step $k^{\textrm{s}}$ that can be occupied by the vehicles that enter the cell and join the idling regime within one simulation sampling cycle, where this percentage depends on the idling speed the vehicles adopt in the cell for this simulation sampling time step and the speed of the congestion wave in the cell

• •

$\xi_{i}(k^{\textrm{s}})$: percentage of the vacant capacity in cell $i$ at the beginning of the current simulation sampling cycle that can be allocated to the vehicles that enter the cell via on-ramp $i$

Remark 2

Note that in the given definitions the term “for the simulation sampling time step $k^{\textrm{s}}$” in the discrete-time domain is an equivalent for the statement “during the sampling time interval $\left[k^{\textrm{s}}c^{\textrm{s}},(k^{\textrm{s}}+1)c^{\textrm{s}}\right)$” in the continuous-time domain, with $c^{\textrm{s}}$ the simulation sampling cycle.

The main equations of the ACTM for updating the state variables of the model, i.e., $n_{i}(k^{\textrm{s}})$ and $q_{i}(k^{\textrm{s}})$, are given by:

[TABLE]

where the mainline outflow of cell $i$ is computed by:

[TABLE]

Note that in the ACTM, the cells are defined in such a way that the mainline inflow, $o_{i-1}(k^{\textrm{s}})$, does not contribute to the mainline outflow, $o_{i}(k^{\textrm{s}})$, of cell $i$ (see (6)), while a percentage of the on-ramp inflow, i.e., $\alpha_{i}(k^{\textrm{s}})e_{i}(k^{\textrm{s}})$, contributes to the mainline outflow of the cell. In other words, all the vehicles in the mainline inflow of cell $i$ blend with the idling flow in the cell. The on-ramp inflow of cell $i$ is computed by:

[TABLE]

where $\mu_{i}(k^{\textrm{s}})$ is the metering rate of the metered on-ramp $i$ for the simulation sampling time step $k^{\textrm{s}}$, i.e., total number of vehicles that are allowed to enter cell $i$ via on-ramp $i$ at the simulation sampling time step $k^{\textrm{s}}$.

Finally, for the off-ramp outflow of cell $i$, we have444In the boundary case where $\beta_{i}(k^{\textrm{s}})=1$, the following equation may be used: s_{i}(k^{\textrm{s}})=\min\bigg{\{}\bar{s}_{i},\left(n_{i}(k^{\textrm{s}})+\alpha_{i}(k^{\textrm{s}})e_{i}(k^{\textrm{s}})\right)\eta^{\textrm{m}}_{i}(k^{\textrm{s}})\bigg{\}}.:

[TABLE]

The stretch of a single-lane highway we use for the case study is illustrated in Figure 10. The highway stretch is divided into 6 cells where the second, fourth, and fifth cells each include one metered on-ramp and one off-ramp. The traffic flow is in the direction of the increase in the cell indices, i.e, it is from left to right in Figure 10. The simulation parameters have been selected based on [21] and in line with [20]. These parameters are listed in Table I. We run the simulations for 180 simulation time steps, i.e., for a period of 1 hours.

The measured values of the demands at the mainstream and on-ramps used for the case study are represented in Figure 11. In order to make the scenarios of the case study more realistic, for the predicted values of the demands at the origin of the mainstream, i.e., $d^{\textrm{M}}$, and at the beginning of the on-ramps, i.e., $d_{2}$, $d_{4}$, $d_{5}$, we randomly add/subtract a random error of up to 10% of the measured values to/from them. The initial states of the network (given in [veh]) are $n_{1}(0)=32.6,\ n_{2}(0)=36.2,\ n_{3}(0)=5.1,\ n_{4}(0)=25.3,\ n_{5}(0)=3.9,\ q_{2}(0)=5.5,\ q_{4}(0)=9.6,\ q_{5}(0)=1.6$.

IV-B Base and parallel controllers

The base-parallel control architecture we have used for this case study is illustrated in Figure 12. In the base block, we have considered an explicit (ALINEA) and an implicit (ANN) base controller, which are detailed next. ALINEA [22] is a feedback-based control approach for ramp metering in road traffic. The control signal produced by ALINEA for the control sampling time step $k$ for the ramp meter of on-ramp $i$ is

[TABLE]

with $\theta^{\textrm{ALINEA}}_{i}(k)$ the tuning parameter of ALINEA for cell $i$ at the control sampling time step $k$, and $\rho^{\textrm{crit}}_{i}$ and $\rho_{i}(k)$ the critical and measured densities (i.e., total number of vehicles per unit length per lane) of cell $i$ at the control sampling time step $k$ directly downstream of on-ramp $i$. For the ALINEA block, we apply the constant gain of $\theta^{\textrm{ALINEA}}=0.016$ from [23], when using the SI units for the variables in (14). Moreover, the values (given in [veh] per simulation sampling cycle) $\mu_{2}=0.5,\ \mu_{4}=0.2,$ and $\mu_{5}=0.4$ are used as the previous control signals in order to evaluate the control signal of ALINEA for the initial simulation sampling time step. For $\rho^{\textrm{crit}}_{i}$, we use $0.0335$ veh/m/lane [24] for all the 6 cells shown in Figure 10. Note that since $\rho_{i}(k)$ is not computed directly as a state variable via the ACTM, we should compute it at every control sampling time step in veh/m/lane, which based on the definition of the density, is determined by $\rho_{i}(k)=n_{i}(k)/(\textrm{cell length})$, reminding that the highway of the case study is single-lane. In Figure 12, the vector $\bm{\mu}(k)$ includes $\mu_{2}(k)$, $\mu_{4}(k)$, and $\mu_{5}(k)$, i.e., the ALINEA block illustrated in Figure 12 includes the control policy of (14) and hence, the tuned parameters of ALINEA for all the three metered on-ramps.

For the implicit base controller, a mapping based on an artificial neural network (ANN) is trained (see Figure 13), which receives the state variables $n_{i}(k)$ and $q_{i}(k)$ and the uncontrolled external inputs $d_{i}(k)$ and $o_{i-1}(k)$ of a cell for every control sampling time step $k$, and produces a value for $\theta^{\textrm{ALINEA}}_{i}(k)$ in (14). The ANN is a feedforward network with one hidden layer of size three. To train this mapping, we have generated a dataset of size $500$, including $n_{i}$, $q_{i}$, $d_{i}$, and $o_{i-1}$ with $i=2,4,5$ for $500$ various traffic scenarios as the inputs of the ANN for cell $i$, and optimized values of $\theta^{\textrm{ALINEA}}_{i}(k)$ as the output. The input values are considered such that a wide range of possible scenarios is covered. To generate the output values, we consider each metered cell as an isolated one, i.e., with infinite capacity for the leaving flows, and minimize the difference between the critical density $\rho^{\textrm{crit}}_{i}$ and the cell’s expected density $\rho^{\textrm{exp}}_{i}(k+1)$ with respect to $\theta^{\textrm{ALINEA}}_{i}(k)$ as the optimization variable. This is because the objective of ALINEA is to keep the road’s density downstream of an on-ramp near the critical density (see (14) and [25]). The expected density is determined by dividing the expected total number of vehicles $n^{\textrm{exp}}_{i}(k+1)$ in cell $i$ by the length of the cell, with

[TABLE]

where $e_{i}(k)$ is computed by the second formulation of (12) and $\mu_{i}(k)$ by (14). The generated dataset is divided into a training dataset including 400 data, and a validation dataset including the rest 100 data. Note that the ANN in Figure 12 includes three mappings for the three ramp meters, i.e., the vector $\bm{\theta}^{\textrm{ALINEA}}(k)$ includes $\theta^{\textrm{ALINEA}}_{2}(k)$, $\theta^{\textrm{ALINEA}}_{4}(k)$, and $\theta^{\textrm{ALINEA}}_{5}(k)$. For the case study, in addition to the initial state of the network and the initial values of the measured demands, we have used $o_{1}(0)=3.8,\ o_{3}(0)=3.2,$ and $o_{4}(0)=0.6$ (given in [veh] per simulation sampling cycle) to evaluate the outputs of the three ANN-based mappings for the initial simulation sampling time step. The output $\bm{\theta}^{\textrm{ALINEA}}(k)$ of the ANN is first transformed into $\bm{\mu}(k)$ using (14) (see Figure 12). The ACTM is then used to generate a sequence $\tilde{\bm{\mu}}$ using the control inputs generated by the ALINEA and the ANN blocks.

For each base controller, we consider a parallel cell with two MPC-based controllers, where for the explicit base controller (ALINEA), the MPC-based controllers (CMPC(1) and CMPC(2) in Figure 12) are formulated using conventional MPC, and for the implicit base controller (ANN), parameterized MPC (PMPC(1) and PMCP(2) in Figure 12) is used. We use the “fmincon” solver with multiple starting optimization points and the Sequential Quadratic Programming (SQP) algorithm from the Matlab optimization toolbox, to solve the MPC optimization problems, which, in general, are nonlinear and non-convex. The function tolerance and the step tolerance are set to, respectively, $10^{-3}$ and $10^{-7}$. The cost function $J(k)$ of the MPC optimization problems at the control sampling time step $k$ is defined by

[TABLE]

with TT and TD the travel time and the traveled distance of all the vehicles. We have used $\gamma=0.8$ for the simulations. The MPC optimization problems are subject to constraints, including the traffic dynamics, upper and lower bounds for the ramp meters, and lower bound for the queue lengths on the on-ramps.

The main difference between the two MPC-based controllers indexed by 1 and 2 in each cell is in the size of their prediction horizons $n^{\textrm{H}}(1)$ and $n^{\textrm{H}}(2)$, i.e., 3 and 10. More specifically, CMPC(1) and PMPC(1) have the prediction horizon $n^{\textrm{H}}(1)$ and CMPC(2) and PMPC(2) have the prediction horizon $n^{\textrm{H}}(2)$. Note that a large prediction horizon provides a more extensive vision of the future, which may help to reduce the negative effects of the finite horizon of MPC on the degree of optimality of the solutions. However, the cumulative errors resulting from the inaccuracies in the prediction model are larger for a larger prediction horizon.

The candidate control inputs produced by ANN, ALINEA, PMPC(1), PMPC(2), CMPC(1), and CMPC(2) are all sent to the evaluation block (see Figure 12), where six ACTM blocks will in parallel estimate the realized value of the cost function of all the vehicles for a prediction time horizon of size $\bar{n}^{\textrm{e}}$ (assuming that $\bar{n}^{\textrm{e}}\leq n^{\textrm{H}}(1)<n^{\textrm{H}}(2)$) for each candidate control input. We have used $\bar{n}^{\textrm{e}}=3$ for the case study. The “Selector” compares these values and selects the control input $\bm{\mu}^{\textrm{best}}(k)$ that corresponds to the minimum cost.

IV-C Results

Table II includes the results of the case study for a 1-hour simulation repeated with different control approaches, Alinea, ANN, CMPC(1), CMPC(2), PMPC(1), PMPC(2), and the base-parallel control architecture given by Figure 12. Since, in general, the optimization problems of the MPC-based controllers are non-convex, the optimization solvers may return a locally optimal solution instead of a global optimum. In order to deal with this problem, we have proposed three different starting points for the MPC optimization solvers, i.e.,

• •

solution of the optimization corresponding to the previous control sampling time step, ignoring the first element of the sequence and repeating the last element twice;

• •

average of the solution of the optimization corresponding to the previous control sampling time step (ignoring the first element of the sequence and repeating the last element twice) and the solution of the optimization corresponding to the second previous control sampling time step (ignoring the first two elements of the sequence and repeating the last element three times), where this starting point can be used for $k\geq 2$;

• •

average of the solution of the optimization corresponding to all the previous control sampling time steps (ignoring those elements of the sequences that correspond to the past control sampling time steps and repeating the last elements to cover the prediction time window).

In Table II for each control approach, we have represented the realized value of the total cost function, $J^{\textrm{total}}$, in [h] for the 1-hour simulation period, which has been computed via (15) per control sampling time step and accumulated across the simulation time window. The total number of vehicles, $n^{\textrm{total}}$, that can travel through the controlled stretch of the road within the fixed simulation time window is another indication for the control performance. In other words, we expect a high-performing control approach to minimize $J^{\textrm{total}}$, while allowing higher numbers of vehicles that intend to enter the stretch of the road to travel via it in the given simulation time window. Therefore, the ratio of these two values, i.e., the average cost per vehicle, can be an indication of the control performance. The lower this ratio, the better the control approach. Hence, in Table II we have also represented $n^{\textrm{total}}$ in [veh] and the average cost per vehicle in [s] (i.e., $J^{\textrm{total}}*3600$ divided by $n^{\textrm{total}}$), for all the control approaches used in the case study.

Based on the results given in Table II, CMPC(1), CMPC(2), PMPC(1), and PMPC(2) result in the lowest values for the average cost per vehicle. Comparing the three real-time control approaches (i.e., they can always produce their candidate control inputs in a time budget smaller than or equal to the control sampling time of the controlled system) ALINEA, ANN, and the proposed base-parallel control architecture, the lowest average cost per vehicle corresponds to the proposed base-parallel control architecture. It is also important to indicate that among all the 7 control approaches considered in this case study, the proposed base-parallel control approach results in the lowest value of the total cost.

V Discussion

The conventional and parameterized MPC approaches have resulted in the best performance among the other control methods used in the case study. This is somehow expected since MPC approaches minimize the cost function at every control sampling time step by solving an online optimization problem, based on the updated measured values of the state and uncontrolled external inputs. The main issue with MPC, however, is the high computation time, which may exceed the control sampling cycle of the controlled system, especially when the scales of the controlled system and the complexity of the dynamics increase. Moreover, due to the non-convexity of the optimization problem, several starting optimization points should be considered to make sure more sub-regions in the optimization search region are covered. This will cause increased computational burden and computation time.

However, when the MPC-based approaches are used in the proposed parallel block, the issue with the computational burden and computation time can be tackled due to the following reasons:

• •

The MPC-based controllers are given a time budget that is smaller than or equal to the control sampling time of the controlled system. Therefore, the control system will not wait longer than this time budget, and in case the control inputs returned by any of the MPC-based controllers result in a non-satisfactory performance, they will be excluded by the evaluation block and the selector.

• •

Several MPC-based controllers will run in parallel, i.e., instead of running one MPC module in a serial set-up with various optimization starting points (as for CMPC(1), CMPC(2), PMPC(1), and PMPC(2)), they will be run in parallel. Therefore, the computational burden is distributed among the parallel MPC-based controllers, which results in reduced computation time.

• •

The existence of the base block next to the online MPC-based controllers provides a warm-start for them that reduces the computational burden and the computation time, which are large for the conventional and parameterized MPC-based controllers, CMPC(1), CMPC(2), PMPC(1), and PMPC(2), due to an extensive exploration of the optimization search region.

Moreover, a detailed assessment of the prediction horizon for real-life processes is not possible prior to running the control procedure, particularly when the nonlinearities increase. Therefore, we allow both a small and a large prediction horizon for the MPC-based controllers via the use of parallel cells within the proposed control architecture.

The lower value of the realized total cost function corresponding to the base-parallel control architecture in comparison with the MPC-based approaches that aim at minimizing the same cost function online at every control sampling time step, together with the fact that for the majority of the control sampling time steps, one of the MPC-based controllers has won the competition in the evaluation block, may be explained via the following reason. The warm-start provided via the base block for the online MPC optimization problems in the parallel block, has improved the performance of these controllers for several control sampling time steps. In other words, although we may provide several starting points for the online MPC optimization problems, this will not guarantee that the entire optimization search region will necessarily be covered. More specifically, this will not guarantee that the sub-regions that correspond to global or better local optima will be covered, while the warm-start provided by the base block has in several cases allowed the online optimizers to search such sub-regions.

VI Conclusions and Future Work

We have proposed an integrated base-parallel control architecture with the aim of addressing the current gap in efficient real-time implementation of MPC-based control approaches for highly nonlinear systems with fast dynamics and a large number of control constraints. In this architecture, several offline tuned or optimized controllers and online optimization-based controllers can run in parallel.

We have performed a simulation for controlling the metered on-ramps of a stretch of a highway. The simulation results have shown that, among the control approaches that can perform in real time, the proposed base-parallel control architecture has the performance (considering the average cost per vehicle) that is the closest to the online MPC-based controllers, which produce optimal control inputs, but suffer from high computation times. Moreover, the least value of the overall cost among all these 7 control approaches, corresponds to the base-parallel control architecture. The new control architecture results in a control system that is very flexible and its architecture can easily be changed or modified online.

Relevant topics that require further exploration in the future include:

• •

Stability analysis of the proposed base-parallel control architecture, assuming individually stable base and parallel controllers.

• •

More extensive simulations including various large-scale controlled systems with highly nonlinear dynamics, and including different base and parallel controllers.

• •

In-depth study and analysis of the influences of different tuning parameters of MPC for various dynamics of the controlled system. This can be done by running several parallel MPC-based controllers in the parallel block and tracking the trend of the selection of each candidate control input based on the ongoing dynamics of the controlled system.

Acknowledgments

This research has been supported by the NWO-NSFC project “Multi-level predictive traffic control for large-scale urban networks” (629.001.011), which is partly financed by the Netherlands Organization for Scientific Research (NWO).