Interpretable PID Parameter Tuning for Control Engineering using General Dynamic Neural Networks: An Extensive Comparison

TL;DR

This paper explores extending traditional PID controllers with General Dynamic Neural Networks to improve control performance on complex systems while maintaining interpretability and stability, validated through extensive benchmarking.

Contribution

It introduces neural PID controllers using GDNNs, demonstrating their superior performance and interpretability compared to standard and model-based controllers across multiple benchmarks.

Findings

Neural PID controllers outperform standard PID in 15 of 16 tasks.

Neural PID controllers outperform model-based control in 13 of 16 tasks.

Bounded-input bounded-output stability analysis enhances interpretability of neural controllers.

Abstract

Modern automation systems rely on closed loop control, wherein a controller interacts with a controlled process, based on observations. These systems are increasingly complex, yet most controllers are linear Proportional-Integral-Derivative (PID) controllers. PID controllers perform well on linear and near-linear systems but their simplicity is at odds with the robustness required to reliably control complex processes. Modern machine learning offers a way to extend PID controllers beyond their linear capabilities by using neural networks. However, such an extension comes at the cost of losing stability guarantees and controller interpretability. In this paper, we examine the utility of extending PID controllers with recurrent neural networks-namely, General Dynamic Neural Networks (GDNN); we show that GDNN (neural) PID controllers perform well on a range of control systems and highlight how they can be a scalable and interpretable option for control systems. To do so, we provide an extensive study using four benchmark systems that represent the most common control engineering benchmarks. All control benchmarks are evaluated with and without noise as well as with and without disturbances. The neural PID controller performs better than standard PID control in 15 of 16 tasks and better than model-based control in 13 of 16 tasks. As a second contribution, we address the lack of interpretability that prevents neural networks from being used in real-world control processes. We use bounded-input bounded-output stability analysis to evaluate the parameters suggested by the neural network, thus making them understandable. This combination of rigorous evaluation paired with better interpretability is an important step towards the acceptance of neural-network-based control approaches. It is furthermore an important step towards interpretable and safely applied artificial intelligence.

Figures

Tables

Table 1. Table 1: Control results for the four benchmark systems.

Control Benchmark		RMSE on test data over 50 runs
Disturbance		-	-	✓	✓
SNR		-	20 dB	-	20 dB
Two-tank system
Mean	Neural PID	$0.74$	$0.86$	$0.84$	$1.00$
	Standard PID	$0.95$	$0.99$	$1.00$	$1.10$
	Backstepping	$1.10$	$1.10$	$1.20$	$1.10$
Variance	Neural PID	$0.0036$	$0.0040$	$0.0056$	$0.0460$
	Standard PID	$0.0025$	$0.0023$	$0.0052$	$0.0063$
	Backstepping	$0.0016$	$0.0018$	$0.0036$	$0.0024$
Inverted pendulum
Mean	Neural PID	$0.034$	$0.02$	$0.09$	$0.27$
	Standard PID	$0.035$	$0.04$	$140$	$140$
	LQ regulator	$0.05$	$0.05$	$140$	$140$
Variance	Neural PID	$0.0004$	$0.0003$	$0.0490$	$0.0007$
	Standard PID	$0$	$1.1\cdot {10}^{-7}$	$0$	$0.0023$
	LQ regulator	$0$	$3.3\cdot {10}^{-9}$	$0$	$0.0022$
LTI system with input delay
Mean	Neural PID	$0.13$	$0.15$	$0.26$	$0.28$
	Standard PID	$0.22$	$0.23$	$0.36$	$0.38$
	Smith predictor	$0.18$	$0.19$	$0.19$	$0.20$
Variance	Neural PID	$0.0006$	$0.0005$	$0.0007$	$0.0003$
	Standard PID	$0.0007$	$0.0006$	$0.0004$	$0.0003$
	Smith predictor	$0.0006$	$0.0005$	$0.0005$	$0.0003$
Chaotic thermal convection loop
Mean	Neural PID	$0.23$	$0.90$	$1.90$	$1.70$
	Standard PD	$0.24$	$1.60$	$13.00$	$4.20$
	Backstepping	$0.26$	$0.89$	$9.80$	$9.80$
Variance	Neural PID	$1.2\cdot {10}^{-5}$	$0.0003$	$0.16$	$0.22$
	Standard PD	$0$	$2.00$	$0$	$1.40$
	Backstepping	$0$	$0.0001$	$0$	$1.4\cdot {10}^{-5}$

Equations

$$J_{\textbf{w}}\big{(}\textbf{v}(\textbf{x};\textbf{w})\big{)}=\begin{bmatrix}\frac{\partial}{\partial w_{0}}v_{0}(x_{0};\textbf{w})&\frac{\partial}{\partial w_{1}}v_{0}(x_{0};\textbf{w})&\cdots&\frac{\partial}{\partial w_{k}}v_{0}(x_{0};\textbf{w})\\ \frac{\partial}{\partial w_{0}}v_{0}(x_{1};\textbf{w})&\frac{\partial}{\partial w_{1}}v_{0}(x_{1};\textbf{w})&\cdots&\frac{\partial}{\partial w_{k}}v_{0}(x_{1};\textbf{w})\\ \vdots&\vdots&\ddots&\vdots\\ \frac{\partial}{\partial w_{0}}v_{0}(x_{p};\textbf{w})&\frac{\partial}{\partial w_{1}}v_{0}(x_{p};\textbf{w})&\cdots&\frac{\partial}{\partial w_{k}}v_{0}(x_{p};\textbf{w})\\ \frac{\partial}{\partial w_{0}}v_{1}(x_{0};\textbf{w})&\frac{\partial}{\partial w_{1}}v_{1}(x_{0};\textbf{w})&\cdots&\frac{\partial}{\partial w_{k}}v_{1}(x_{0};\textbf{w})\\ \frac{\partial}{\partial w_{0}}v_{1}(x_{1};\textbf{w})&\frac{\partial}{\partial w_{1}}v_{1}(x_{1};\textbf{w})&\cdots&\frac{\partial}{\partial w_{k}}v_{1}(x_{1};\textbf{w})\\ \vdots&\vdots&\ddots&\vdots\\ \frac{\partial}{\partial w_{0}}v_{1}(x_{p};\textbf{w})&\frac{\partial}{\partial w_{1}}v_{1}(x_{p};\textbf{w})&\cdots&\frac{\partial}{\partial w_{k}}v_{1}(x_{p};\textbf{w})\\ \vdots&\vdots&\ddots&\vdots\\ \frac{\partial}{\partial w_{0}}v_{q}(x_{0};\textbf{w})&\frac{\partial}{\partial w_{1}}v_{q}(x_{0};\textbf{w})&\cdots&\frac{\partial}{\partial w_{k}}v_{q}(x_{0};\textbf{w})\\ \frac{\partial}{\partial w_{0}}v_{q}(x_{1};\textbf{w})&\frac{\partial}{\partial w_{1}}v_{q}(x_{1};\textbf{w})&\cdots&\frac{\partial}{\partial w_{k}}v_{q}(x_{1};\textbf{w})\\ \vdots&\vdots&\ddots&\vdots\\ \frac{\partial}{\partial w_{0}}v_{q}(x_{p};\textbf{w})&\frac{\partial}{\partial w_{1}}v_{q}(x_{p};\textbf{w})&\cdots&\frac{\partial}{\partial w_{k}}v_{q}(x_{p};\textbf{w})\end{bmatrix}\in\mathbb{R}^{pq\times k}$$

$$\hat{J}_{\textbf{w}}\big{(}\textbf{v}(\textbf{x};\textbf{w})\big{)}=\begin{bmatrix}\textbf{j}_{0}\\ \textbf{j}_{1}\\ \vdots\\ \textbf{j}_{k}\end{bmatrix}^{T}\,,\,\textbf{j}_{i}=\frac{\textbf{v}(\textbf{x};\textbf{w})-\textbf{v}(\textbf{x};\textbf{w}-\eta\varepsilon(\textbf{w}_{i}))}{\varepsilon(\textbf{w}_{i})}\,,\,\eta_{p}=\left\{\begin{array}[]{ll}1,&p=i\\ 0,&p\neq i\end{array}\right.,$$

$$\epsilon(\textbf{w}_{i})=\max\big{(}1,|\textbf{w}_{i}|\big{)}\sqrt{\epsilon_{\text{min}}},$$

$$\begin{array}{rl}\dot{x_1}(t)& =p(x_2-x_1),\\ \dot{x_2}(t)& =x_1-x_2-x_3(x_1+\beta ),\\ \dot{x_3}(t)& =x_1{x}_2+\beta (x_1+x_2)-x_3-u,\end{array}$$

$$\frac{V(s)}{V^{\ast }(s)}=\frac{G(s)H(s)}{1+G(s)H(s),}$$

Full text

1 Introduction

Modern production engineering involves increasingly complex physical processes [1]. The physical processes underlying cutting-edge production engineering cannot be appropriately expressed with simple models [2]. New, more complex classical control methods are being developed, but their increased complexity is what makes them challenging for control engineers to design and apply[3]. Limited by both the number of skilled control engineers and their cost, the production engineering industry has widely chosen to continue to use simple, understandable linear controllers. While these controllers are easy to set-up and adjust, they are not suitable for the complex non-linear behaviour of the processes they are expected to control. Using these simple controllers comes at a cost: it means processes will require close monitoring and human assistance whenever the system changes in an unforeseen way. In the face of increasingly complex systems, both control-engineer-designed methods and closely-monitored simple controllers fail to scale.

One way to bridge this gap between simple controllers and complex control systems is by applying modern machine learning techniques [4]. Extending the capabilities of well-accepted and used controllers with machine learning yields a potential solution to the lack of scalability and adaptability in existing control approaches. In this paper, we will investigate the use of neural networks to adapt the parameters of a Proportional-Integral-Differential (PID) controller not only before deployment but online during ongoing control. This continuous adaptation to the process would allow for linear controllers to perform well for any control task, as the controller constantly linearizes its behaviour around the state the system is currently in. As our investigation shows, this results in superior control performance and disturbance rejection.

The PID controller is one of the most widely used controllers [5]. A PID controller calculates its control output $u$ based on the current error $e$, the error derivative $\frac{de}{dt}$ and the error integrated over time $\int_{0}^{t}edt$. Each error measure is multiplied by its corresponding constant—$K_{P}$ with the error, $K_{I}$ with the integrated error, and $K_{D}$ with the error derivative—and then summed. A PID controller is tuned with respect to the system to be controlled by adjusting these three constants. While this controller is simple and well understood, its advantages come at the price of limited capabilities. PID controllers perform well only on linear systems or systems that are linearized. As PID controllers are usually adjusted before deployment, they neither handle disturbances nor varying system dynamics well enough to meet the needs of modern production engineering.

Several prior studies have shown that neural networks can improve performance when used to tune the parameters of traditional PID controllers online. Simple feed-forward networks can adjust the PID parameters in multiple settings [6, 7]. More sophisticated neural networks like strictly recurrent, diagonal [8] and quasi-diagonal [9] recurrent networks have also been investigated. However, these specific design choices for the neural network’s architecture result in very specific behaviour and learning [10] and thus are unlikely to generalize well over different control environments. To evaluate whether the use of neural networks with all possible connections is applicable to control engineering tasks in general, a broad range of different control environments is required. In this paper, we therefore implement and evaluate general dynamic neural networks, thus extending the current state of the art.

Previous works have demonstrated the applicability of neural-network-based parameter tuning, but only for one well-defined control task at a time, e.g. pendulums [7], two-tank systems [11] or magnetic systems [6]. As no single approach has been applied successfully to multiple representative control engineering problems in the literature, the effectiveness of the suggested approaches is hard to estimate. For this reason, we investigate our approach on four different systems that represent the most common challenges in control engineering, resulting in an extensive comparison. By comparing the control performance on all four different tasks, we extend the current literature which focused on one problem at a time. Additionally, our simulations are closer to real-world conditions than those used in previous work, as they include disturbances and noise.

Despite their potential to improve control performance, neural networks are not widely used for parameter tuning in real-world control systems. One major barrier prevents their adoption: due to the lack of interpretability, the suggested PID parameters cannot be evaluated for their appropriateness. As a result, the control loop itself loses one of its most important properties—its stability guarantees. Previous papers that utilized neural networks for PID parameter tuning did not address the effect on input-output stability, thus ignoring a key concern in control engineering. The domain of control engineering demands interpretability to ensure system safety [12]. As neural networks introduce a black box to the system [13], their use for PID parameter tuning is not yet widely accepted. Real-world use requires the ability to check and reason about the parameter choices a neural network outputs when granted access to a PID controller’s parameters. We suggest that the use of neural networks will in fact enable designers to achieve better performance without significantly increasing the design complexity or diminishing design interpretability when deploying a controller—if combined with traditional control engineering tools like stability analysis.

To summarize our contributions: as a first contribution of this paper, we advance the use of specifically tailored neural network architectures by investigating the use of General Dynamic Neural Networks (GDNNs) for online parameter tuning in PID controllers. These neural PID controllers are evaluated on four different closed loop control engineering tasks. Each task represents a different, common challenge in control engineering, namely non-linear behaviour, unstable equilibrium, dead time, and chaotic behaviour. These applications have furthermore been used for evaluation of control at several machine learning venues, e.g. [14, 15, 16]. We compare the performance of our neural approach with a standard PID controller, which acts as a baseline, and with a system-appropriate model-based controller, which should provide the best possible performance. To evaluate the robustness of our approach, each comparison is performed with and without significant sensor noise and with and without disturbance. All controllers are evaluated quantitatively for all scenarios, making this study unique in its comprehensiveness when compared to prior work.

As a second contribution, we demonstrate how input-output stability analysis—a classic analysis that is well known in control engineering—can be used in a novel, online way to explain the effects of parameter tuning. This analysis is done for the control system, exhibiting chaotic behaviour as an example of a challenging control task. This assessment provides an explanation of when the system is stable with respect to the PID parameters, making the neural network outputs understandable to control engineers. While input-output stability is an important aspect of every closed-loop control approach, it is widely ignored in the existing literature that suggests the use of neural networks for PID parameter tuning. This paper therefore addresses key attributes of safe and interpretable artificial intelligence.

2 General Dynamic Neural Network for PID Parameter Tuning

The aim of this work is to extend the classic PID controller framework, composed of the PID controller and the plant, with as few changes to the physical setup of the control system as possible. Maintaining the core structure of the classic PID control framework allows for easy adoption to existing industrial applications. This structure is preserved by restricting the neural network’s inputs to signals which are already available in the closed loop control setting. The closed loop control setting is depicted in Figure 1(a). The neural network’s inputs are the control system’s output $v$ and the control error $e$, while the outputs are the three PID parameters to tune, i.e. $K_{P}$, $K_{I}$, and $K_{D}$. At each time step, the neural network computes the PID parameters based on the observations and passes them into the PID controller. The PID controller then uses these parameters to compute the control output $u$.

To ensure fast online computation with limited hardware, the neural networks implemented in this investigation are restricted to one hidden layer with four neurons. Furthermore, for control systems that can be described by simple differential equations increasing the number of neurons would lead to overfitting. The network architectures differ only in additional recurrent or feedback connections. An example can be seen in Figure 1(b). The figure shows a neural network, where the output of each neuron in the first layer is fed back as part of the first layer’s input with a delay, denoted as $g^{-1}$, of one time step. For all neurons, the activation function tanh is chosen, following the rationale of [17].

The standard approach for training neural networks is backpropagation [18]. Most deep learning approaches adjusts the neural network’s weights by end-to-end optimization [19]. This optimization involves formulating a loss function that describes the difference between the neural network’s outputs and the ideal outputs. From this loss, a gradient with respect to the weights is computed and propagated through the network. At each neuron, the weights are then adjusted to minimize the loss. However, in the present framework, Figure 1(a), backpropagation cannot be naively applied. In this framework, the neural network’s outputs are the PID parameters $K_{P}$, $K_{I}$, and $K_{D}$. Using standard backpropagation would therefore require knowing the ideal PID parameters at any given time.

A way to train the neural network without the ideal outputs is to numerically approximate backpropagation. In this work, we chose a numerical Levenberg-Marquard algorithm [20] to minimize the squared control error. At each time step, the Jacobian matrix, shown in Equation 1, of the neural network weights is numerically approximated using finite differences with respect to small changes in each weight. Each weight is then adapted to decrease the control error.

The Jacobian matrix of a general dynamic neural network with $p$ input neurons, $q$ dynamical system (plant) outputs and k weighted connections was calculated as

[TABLE]

, where $\textbf{x}\in\mathbb{R}^{p}$ is the neural network input vector, $\textbf{w}\in\mathbb{R}^{k}$ is the vector of weights which describes the network topology and $\textbf{v}(\textbf{x};\textbf{w})\in\mathbb{R}^{q}$ are the outputs of the dynamical system (plant). According to [21], it is sufficient to calculate the partial derivatives of the systems output instead of the error function.

As the analytic calculation would result in extensive computations, it is numerical approximated using a difference equation, rather than a differential one as solution of the equation

[TABLE]

where $\hat{J}_{w}$ is the approximated Jacobian matrix, $\textbf{j}_{i}$ is the $i$-th column of $\hat{J}_{w}$, $\eta$ is the step size and $\epsilon$ is the machine precision. $\epsilon$ has to be calculated for each pass, using the equation

[TABLE]

where w is a vector, containing the weights, and $\epsilon_{\text{min}}$ is the implementation data type, double precision in this implementation. The complete calculation of the Jacobian matrix can be found in Algorithm 1.

In the industrial control setting we consider in this paper, it is important that finding an appropriate network architecture, i.e. the connections and delays between neurons, does neither need sophisticated engineering nor significant time. Any approach that makes the set up too complicated would defeat the purpose of extending the existing PID framework. We therefore chose to create ten neural network architectures for each control challenge by randomly adding feedback connections and then chose the neural network that performed best. This process of finding an architecture can be made more efficient by using search algorithms [22]. For each tested neural network, the weights were initialized randomly with a mean of zero and a standard deviation of one.

Training data was collected from the differential equations, that describe each benchmark system, without any additional noise or disturbances. We followed an approach from classical control engineering by exciting the system with input signals of different lengths and amplitudes—called Amplitude Modulated Probabilistic Random Binary Signals (APRBS) [23]—to collect data samples that sufficiently describe the dynamics of the system [24]. This approach is an extension of using a Dirac impulse for system identification [25]. We collected $35,000$ data samples for each benchmark system, and split them into training, validation, and test sets with a ratio of $15\%$, $30\%$, and $55\%$, respectively.

3 Experiments

To evaluate the performance of the neural PID controller, we use four typical control problems. Each system offers a different control challenge. Each individual system is controlled with and without noise. The noise is Gaussian white noise with a signal to noise ratio (SNR) of $20$dB and corresponds to noisy measurements from sensors and is added to the system output.

To further evaluate the robustness of the controller, we inflict upon each system a suitable disturbance. The term disturbance refers to an unwanted and unexpected system input that will result in an increase of the system error. Disturbances can occur due to external influences or due to failure in the system and they are individual for each kind of system. For each system, the disturbance is increased in size from zero until only one control approach is still able to stabilize the system. Disturbances of this magnitude are then used in the experiments. These disturbances are not included in the training data for the neural PID controller. All experiments are run over $50$ independent runs to ensure statistical relevance. The system differential equations are solved using the Dormand-Prince solver. To simulate real-world conditions of a discrete sample time, the controller output can be adapted every $0.01$s. This intervention time is chosen to represent the limitations of real-world actuators, which cannot adjust their values on an arbitrarily small timescale. The solver is run iteratively for $0.01$s, using the result of the former step as starting conditions. During each $0.01$s intervention time window, the controller output $u$ is kept constant.

The training is done on an Intel Core i5-4570 with a $3.2$ GHz clock rate, $6$ MB of shared L$3$ cache, $32$ GB DDR3 RAM. Once learned, the neural networks run on a Raspberry Pi $3$ Model A+.

Two-Tank System: The first system is a nonlinear two-tank system, as seen in Figure 2 and is described by the differential equations in [26]. The controller has access to a pump, regulating the input, while the measured output is the water level in the second tank. This system is a standard benchmark system in control theory. It corresponds to various industrial processes, e.g. bio-reactors, filtration, and nuclear power plants. There exist a number of control approaches for this system, including direct control via neural networks [27], adaptive output feedback [28], and backstepping [26], which will be used as comparison.

To evaluate the robustness of the compared control approaches, the two-tank system is disturbed continuously between $t=20$s and $t=40$s. As a disturbance, the controller output is set to zero, which would correspond to a drain of the water supply. The water levels in the tank are therefore independent from the control inputs for $20$s. The voltage for the pump, $u$, was limited to the range $[-500\text{V},500\text{V}]$ to simulate a pump appropriate to the tank dimensions.

Inverted Pendulum on a Cart: The second system is a nonlinear inverted pendulum on a cart, as shown in Figure 3. The system is described by the differential equations in [29]. The control task is to stabilize the inverted pendulum at its unstable equilibrium by applying a force on the cart. The cart’s movement is restricted to $0.5$m in either direction. This system is a widespread benchmark system in control theory due to its nonlinearity and unstable equilibrium [29, 30]. Practical applications for inverted pendulums include rocket control during initial stages of flight or keeping a walking robot in an upright position. For comparison, a linear–quadratic regulator (LQR) [31, 32] and a double PID controller [32] are used.

The system is disturbed by a force of $8.5$N to the pendulum at time $t=10$s. This disturbance can be interpreted as a strong and unexpected wind condition during the launch of a rocket. The controller ouptut $u$ was bounded within $[-50\text{N},50\text{N}]$, which corresponds to a typical actuator of that size.

System with Non-negligible Time Delay: The third system is a first order linear time invariant (LTI) system with a non-negligible time delay. Time delay is a problem in control theory that is often forgotten while designing controllers [33]. Time delays can result in decreased performance and system instability. The benchmarks for this system are a PID controller [34] and a smith-predictor [35]. Figure 4 demonstrates the delayed system response for an input.

This system is disturbed by a (dimensionless) disturbance of $-5$ between $t=50$s and $t=75$s continuously. Such a disturbance can be thought of as a temporary blockage in a fluid transport system. The exact system specifications can be found in [34]. The controller output $u$ was bounded between the range of $[-10,10]$.

Chaotic Thermal Convection Loop: The fourth system is a chaotic thermal convection loop, as shown in 5. Its dynamics are described by the equations

[TABLE]

with $p=10$ and $\beta=6$ as appropriate constants [36]. $x_{1}$ is a measure for how far the current flow velocity differs from the steady point of the system—if $x_{1}$ is zero, the system is in its steady point. $x_{2}$ and $x_{3}$ are measures for the difference in temperature between the points A and B, as well as C and D in Figure 5 respectively. $u$ is a the power, applied to the heater and the control variable.

Chaotic behavior may lead to vibrations, oscillations and failure in systems and is therefore an important aspect of control theory. As chaotic behavior is unpredictable, mathematical models are only sufficient to a certain point, hence closed loop control is a desirable approach [37]. Usual control approaches for the chaotic thermal convection loop are nonlinear feedback controllers [38] and backstepping [39, 40].

To evaluate the robustness of the applied control approaches, the system is disturbed with a force of $100$W continuously between $t=5$s and $t=5.5$s. This perturbation can be interpreted as a temporary change in the cooling water temperature. To simulate real actuators with a limited capacity, the controller output $u$ is limited between $[-100\text{W},100\text{W}]$ to simulate an appropriate heating element.

4 Results and Discussion

The results for all experiments can be found in Table LABEL:tab:ctrl_results. For each system the mean and variance from $50$ independent runs are shown for all controllers in all tested scenarios. The best control approach is highlighted in bold. For all values, a two-sample t-test was performed and a control approach is only considered to be superior for $p<0.05$. From Table LABEL:tab:ctrl_results, it can be seen that the neural PID controller performs best in $12$ scenarios, pairs with the standard PID controller in one and performs second best in two scenarios. To compare control approaches, there are common measurements that are used in control engineering, e.g. rise time, overshoot, settling time [5]. However, these measurements all address the error between the setpoint and the actual systems output with different emphases. We therefore used the root-mean-squared-error (RMSE) between the setpoint and the system output to summarize these error measures in a single number without losing information.

Two-Tank System: For the two-tank system, a backstepping controller is chosen for a comparison, as this approach takes the nonlinear behavior of the system into account and has demonstrated to be well suited for this system [26]. The PID controller is parameterized with the constants $K_{P}=3.65,K_{D}=-2$ and $K_{I}=0.4$. All controllers are able to control the system, while the neural PID controller exhibited the smallest error for all scenarios. However, the advantage the neural PID controller yields is relatively small, as can be seen in Table LABEL:tab:ctrl_results. As this system is the easiest to control, it can be expected that the standard PID controller and backstepping perform on a similar scale.

Inverted Pendulum: The inverted pendulum on a cart is controlled by a standard PID controller stack [29], and a LQR [32] for benchmarking. The PID controller, responsible for the position, has the values are $K_{P}=-2.4,K_{D}=-0.75$ and $K_{I}=-1$ and the controller for the angle is set to $K_{P}=25,K_{D}=3$ and $K_{I}=15$ [29]. All three approaches are able to initially stabilize the system. For the scenario without disturbance, the neural PID controller performs equally to the standard PID controller when there is no noise present. For the scenario with only noise, the neural PID controller is the best control approach. Furthermore, only the neural PID controller is able to stabilize the disturbed system, resulting in a substantially lower error as can be seen in Table LABEL:tab:ctrl_results.

System with Non-negligible Time Delay: For the system with non-negligible time delay, the standard PID controller is set to $K_{P}=1.5,K_{D}=-0.1$ and $K_{I}=0.7$. The neural PID controller is superior in all scenarios, when compared to the standard PID controller. When compared to the Smith predictor [41], the neural PID controller performs better only for the scenarios without noise. However, as the Smith predictor has knowledge about the exact time delay, it has a significant advantage over the neural PID controller.

Chaotic Thermal Convection Loop: The PID parameters yielding the lowest error for the chaotic thermal convection loop are $K_{P}=25.3$, $K_{D}=8.9$ and $K_{I}=0$—the controller is therefore a PD controller. The system is initialized outside of its inherently stable region (region of attraction) with the initial conditions $x_{1}=x_{2}=x_{3}=5$. Without control, the system will therefore not converge to the desired steady state $x_{1}=x_{2}=x_{3}=0$.

All controllers are capable of stabilizing the system, as can be seen in Figure 6(a). The backstepping approach has the least overshoot but takes a long time to reach the steady state. The standard PID controller is more aggressive, resulting in a higher overshoot but still a smaller error. The neural PID controller performs best as it finds a good balance between settling time and overshoot.

Out of four scenarios, the neural PID controller demonstrates superior control performance. Only in the scenario with noise but without disturbance does backstepping perform slightly better ($0.89$ vs $0.9$). This can be explained by backstepping being designed using the differential equations. It therefore knows the underlying systems dynamics and is less influenced by the sensor noise.

Between the time $t=5$s and $t=5.5$s, the control output is set to $u=-100$W to simulate the disturbance described earlier. The standard PD controller becomes meta stable and its controller output iterates between the maximum value of $100$W and the minimum value of $-100$W. Although backstepping is proven to be globally, asymptotically stable in the Lyapunov sense [42], it also becomes meta stable. This can be explained by the real world conditions. As the controller can change its control output only every $0.01$s the backstepping approach fails, resulting in switching inputs between the maximum value and the minimum value, as seen in Figure 6(b). Both controllers (PID and backstepping) use excessive amounts of energy without being able to stabilize the system.

The neural PID controller is able to stabilize the system after the disturbance. Figure 6(c) shows how the neural network changes the PID parameters in response to the system output. When $x_{1}$ is far from the setpoint, the $K_{P}$ parameter has a high absolute value to force the system towards its steady state. To further increase the controller output at $t=7.9$s, where the system reaches its furthest distance from the set point $K_{I}$ is increased. After the system reaches its steady state again, all PID parameters are adjusted back to their stationary value to ensure asymptotic performance. The neural PID controller furthermore uses significantly less energy to control the system.

Investigating the solutions stability: Despite the experimental evidence that suggests the enormous benefit of using neural networks to adapt PID parameters online, this approach is not yet used on real-world system. This is due to the black box character of neural networks and the stringent safety requirements for control processes. One of the most important safety requirements of a closed loop control approach is input-output stability. It describes whether the system output is bounded for all bounded inputs. A system can be evaluated for stability by analysing the closed loop transfer function, i.e. the relation between the system output to its input.

For a bounded-input bounded-output stability analysis, the closed loop transfer function is computed as

[TABLE]

where $V(s)$ is the system output, $V^{*}(s)$ is the setpoint, G(s) is the system transfer function and H(s) is the controller transfer function in the Laplace domain. Following the Nyquist criterion, the system is stable if all poles are in the left half of the left half plane, i.e. their real values are smaller than zero. In the example of the chaotic thermal convection loop, the systems transfer function $G(s)$, linearized around a steady state $x_{1}=x_{2}=x_{3}=0$, does not change over time and only has to be computed once. The controller transfer function $H(s)$ is dependent on the PID parameters and therefore changes at every time the neural network adjusts these parameters. To make sense of these changes and interpret them from a stability perspective, the controller transfer function therefore needs to be computed at every time step. This varies from the traditional stability analysis, which is computed once under the assumption of non-changing PID parameters. Together, these transfer functions express whether the closed-loop solution is stable or unstable.

As an important contribution, we therefore perform an online analysis of the input-output stability for the controller. This analysis can be seen in Figure 7. The Figure shows the systems output and the stability, with respect to its linearized steady state over the experiment and the real values for all four poles of the transfer function. The closed loop transfer function is not stable in the beginning, during settling and after the disturbance. This can be expected, as the system is far away from its steady state for which stability is evaluated. However, as the systems output gets close to the set point, i.e. the steady state, the closed loop transfer function becomes stable. Knowing about the relationship between chosen PID parameters and stability allows to include this knowledge into the training. A potential way to include this information would be to include the poles as a regularization term during training in order to force the system towards an input-output stable behaviour. Furthermore, the input-output stability evaluation is an important insight for control engineers and makes the neural PID controllers understandable for humans, thus emphasising its applicability for safety critical systems.

5 Future work

This paper presents a first step towards accommodating the needs of control engineers when integrating machine learning algorithms into existing control architectures. While we identified a way to relate the parameters applied by the neural network back to input-output stability, this new information was not yet leveraged during the training procedure. It is a natural extension of this paper to use the newly found information about stability as a regularizer when training the neural network to ensure only input-output stable PID parameters. Beyond extensions of the exact framework used in this paper, we have provided an effective demonstration of the more general idea of leveraging machine learning algorithms to enhance existing control methods.

While other examples exist in the literature (e.g., the combination of neural networks[43] or fuzzy logic systems [44] with backstepping), many other combinations of different machine learning approaches with other control algorithms have the potential to provide good results. For example, the machine learning methods from reinforcement learning might be well-applied to linear quadratic regulators while maintaining interpretability.

6 Conclusion

In this paper, we conduct an extensive and rigorous investigation into the use of general dynamic neural networks for online PID parameter adaption. We perform experiments on four different systems, with and without sensor noise as well as with and without disturbance, resulting in $16$ experiments in total. These scenarios cover the most important challenges in control engineering. This study is therefore unique in its extensiveness, as previous papers only used one type of benchmark system. The neural-network-based approach outperforms a standard PID controller in $15$ of $16$ scenarios and outperforms a model-based controller in $13$ of $16$ scenarios.

These results showcase the potential of extending existing systems by machine learning in general and neural networks in particular. Furthermore, we keep the neural network design and integration simple to allow for easy adoption of our technique. With an appropriate implementation as a library, our technique could be used without extensive knowledge of either control engineering or neural networks. To the best of our knowledge, this is the first investigation that uses general dynamic neural networks, extending the state of the art for using neural networks to tune PID parameters. We perform a detailed analysis for one representative scenario, highlighting the superior control performance of our approach over both the traditional PID controller and model-based backstepping. It is worth noticing that while training data was gathered from simple differential equations, the results indicate significantly increased resilience towards noise and unforeseen disturbances.

Although the significant potential of neural networks for PID parameter tuning is known [7, 8], this technique has not been used in real-world applications to date. As the functioning of a neural network in this setting is not understood, control engineers refrain from using them. In a first attempt to solve this problem, we perform an input-output stability analysis to interpret how neural networks function within the suggested framework. Tying the neural network outputs back to stability makes this neural-network-based approach understandable to humans. We therefore address a key issue when applying machine learning algorithms for control problems: the interpretability of and subsequently the trust in the machine learning solution. This work is thus an important step to increase the acceptance of machine learning based approaches for real-world systems and an important step towards safe and interpretable applied artificial intelligence.

Acknowledgements

The authors also thank Omid Namaki, Matthew Schlegel, Ahmed Shehata, Brian Tanner, and Nadia M. Ady for suggestions and helpful discussions.