Software principles and concepts applied in the implementation of cyber-physical systems for real-time advanced process control

TL;DR

This paper discusses software principles like shared data, timers, threads, and network communication in implementing cyber-physical systems for real-time process control, demonstrated through Linux and Python examples and a simulation.

Contribution

It provides practical guidance on applying fundamental software principles in CPS for RT-APC, including implementation details for Linux and Python environments.

Findings

Effective implementation of CPS principles demonstrated in Linux and Python

Simulation results validate the proposed software design approach

Highlights importance of network communication for remote control

Abstract

Cyber-physical systems (CPSs) for real-time advanced process control (RT-APC) are a class of control systems using network communication to control industrial processes. In this paper, we use simple examples to describe the software principles and concepts used in the implementation of such systems. The key software principles are 1) shared data in the form of a database, files, or shared memory, 2) timers and threads for concurrent periodic execution of tasks, and 3) network communication between the control system and the process, and communication between the control system and the internet, e.g., the cloud to enable remote monitoring and commands. We show how to implement such systems for Linux operating systems applying the C programming language and we also comment on the implementation using the Python programming language. Finally, we present a complete simulation experiment using a real-time simulator.

Full text

I Introduction

Real-time advanced process control (RT-APC) enables the automatic, reliable, and efficient operation of complex industrial process systems. RT-APC relates to digital control methods such as proportional-integral-derivative (PID) control and model predictive control (MPC) executed periodically with real-time interval timers [1]. Cyber-physical systems (CPSs) combine automatic control with network communication principles for remote control and monitoring of distributed processes. A CPS for RT-APC (CPS-RT-APC) consists of numerous modules, e.g., controllers, user interfaces, and network communication modules that exchange data periodically and concurrently [2, 3]. Consequently, implementations of such systems require real-time interval timers and threading principles as well as shared data and network communication concepts [4, 5]. Additionally, interval timers and threading principles enable the construction of real-time simulators as proxies for the physical plants. Therefore, these software principles enable real-time closed-loop simulations.

Software tools for simulations and development of real-time networked control systems exist: [6] applies TrueTime, a MATLAB/Simulink-based simulation toolbox for real-time networked control systems, to a double tank system and [7] describes real-time enhancements to Etherware, a framework for networked control systems, and applies this framework to an experimental setup of an inverted pendulum. Additionally, [8] describes an industry initiative called Open Process Automation that defines a standards-based, open, secure, interoperable process automation architecture with an objective to provide the computing platforms for the improvement of industrial use of MPC and machine learning based process technologies. However, the multidisciplinary nature of a CPS-RT-APC still makes it a challenging task to implement despite the existence of these software tools and standards, and the descriptions of the key principles of such a system in [1, 4, 5].

In this paper, we demonstrate the key software principles and components in the implementation of a CPS-RT-APC. These software principles and components are 1) shared data, 2) timers and threads, and 3) network communication. We implement the shared data as databases, files, or shared memory. We apply the timers and threading principles for the concurrent and periodic execution of tasks. Finally, we apply network communication between the control system and the plant, and between the control system and the internet for cloud computing and remote monitoring. The novelty of our paper is the explicit demonstration of these software principles and components through code examples. We apply the C programming language in Linux operating systems (OS) and we also comment on the implementation using the Python high-level programming language. Finally, we present a simulation experiment for a CPS-RT-APC applied to a real-time simulator.

The remaining parts of this paper are structured as follows. Section II provides an overview of the CPS-RT-APC software architecture. Section III presents the shared data for such a framework. Section IV describes the real-time interval timers and threading principles. In Section V, we describe the client-server architecture for network communication. Sections VI and VII present implementations of a control module and a real-time simulator. Section VIII comments on the implementation of a user interface for the CPS-RT-APC. Section IX presents a simulation experiment for the CPS-RT-APC framework. Finally, Section X presents conclusions.

II Software Architecture

The CPS-RT-APC software architecture consists of modules that exchange data with shared data periodically. A module executes periodic tasks with a real-time interval timer and we execute the tasks of numerous modules concurrently by applying threading principles. The CPS-RT-APC framework applies a client-server network communication architecture for remote control and monitoring of physical plants. Such a client-server architecture also enables communication between the control systems and the internet, e.g., cloud computing. Additionally, interval timers, threading, and network communication principles enable the implementation of real-time simulators. We connect the servers to the physical plants and simulators, and we implement client modules with real-time interval timers for the periodic transmission of data. Fig. 1 presents a schematic diagram of the CPS-RT-APC software architecture with shared data and modules for a user interface, a controller, and a client. This client module connects to either a physical plant or a real-time simulator.

III Shared Data

We implement shared data as a collection of tables for sensor measurements, actuator commands, and setpoint values and we use $y$, $u$, and $\bar{z}$ to represent these, respectively. We also represent control and operational configurations as tables. We may apply database systems, files, or shared memory to implement these tables. A single table represents the historical data of a unique variable and for each row in a table, we include a time stamp, a status code/description, and a numerical value. We apply Unix time to represent the timestamp and we order the data in the tables in time. Fig. 2 illustrates tables for sensor and actuator data as well as setpoint and control configurations. We apply mutual exclusion (mutex) principles to prevent a module from accessing a table in the shared data when another module is using it. We demonstrate the interaction with shared data using the PostgreSQL database system. Such a database system implements mutex-like principles. Consequently, we do not demonstrate mutex principles explicitly. For these demonstrations, we implement the source code apcshareddata for wrapping libpg-dev, a PostgreSQL C development library, into the functions insertIntoTableFloat() and readMultiRecentValsFloat() for writing and reading of tables, respectively. We use these functions in the real-time timer callback functions, as shown in the later sections.

IV Timers and Threading

Fig. 3 illustrates the concurrent and periodic execution of tasks for the control and client modules in Fig. 1 applying interval timers with thread invocation for the callback functions.

We may implement such interval timers with hard real-time systems principles, e.g., real-time timers with high timing determinism and deadline guarantees. However, such systems rely on special-purpose software such as real-time operating systems (RTOS) [1]. Alternatively, we may apply general-purpose operating systems (GPOS) such as the Linux OS for CPS-RT-APC systems that do not require such high timing determinism. We refer to such systems as soft real-time systems and these will experience larger varying delays (jitter) compared to the hard real-time systems.

We demonstrate the implementation of soft real-time system interval timers for the periodic and concurrent execution of user-defined tasks by applying timers from the portable operating system interface (POSIX) application programming interface (API), defined in POSIX.1b real-time extensions for Linux OS. We configure the timers to invoke their callback functions in new threads to achieve concurrency of the timers [9]. We wrap these POSIX interval timers into functions for creating, starting, and stopping timers and we use structs to pass functions and function arguments to the callback functions through the sigevent structure. Listing 1 presents an example for creating, starting, stopping, and deleting a POSIX interval timer with an interval of 2.5 s. Listing 2 shows the apctimer source code used to create, start, and stop POSIX timers.

IV-A Python implementation

We use Python’s Timer class to schedule the execution of tasks. This class enables the user to schedule the execution of these tasks in separate threads, similar to the POSIX interval timers configured with thread invocation of the callback functions. We create an instance of the Timer class as timer = Timer(interval, func, args=None, kwargs=None), where func is the callback function and interval is the start delay. We achieve periodic execution of the callback function by recursively applying the timer.start() and we apply timer.cancel() to stop the timers[10]. Alternatively, we may design interval timers by applying the sleep function from the time module together with loops. An implementation of such a method without accumulation of time drift is presented in [5].

V Network Communication

Industrial control systems apply network communication to transmit periodic and event-based signals such as control signals, measurements, and alarms between field devices, digital controllers, and operator interfaces. Network communication standards such as the open platform communications unified architecture (OPC UA) standardize the communication between industrial components and many industries apply this standard. The OPC UA standard applies a client-server communication architecture and offers a variety of transport protocols such as TCP/IP based protocols for reliable data transmission [11, 3].

In this paper, we limit the discussion to the periodic transmission of data between the plant and control system by applying a client-server communication architecture with a TCP/IP protocol. We present such a communication architecture for the CPS-RT-APC framework in C with the internet domain stream sockets applied to a real-time simulator. We apply the functions packMultiMsg() and unpackMultiMsg() defined in the source code apcsockets to pack and unpack data from numerous processes into single strings and we transmit these strings between the client and the server. We also apply this library for the initialization of the clients and servers. We do not show the source code but [9] provide code examples for the initialization of these internet domain stream socket clients and servers and how communication between them can be made. We apply real-time interval timers for the client module for the periodic transmission of this data and we pass a pointer to a shared struct between the server and the real-time simulator.

V-A Client

Listing 3 demonstrates how to initialize and connect a client to a server. We demonstrate how to apply an interval timer for the periodic transmission of information between the shared data and the server. The timer callback function is client() and it requires connections to the shared data and to the server. We pass these as arguments to the callback function by using the struct Args_client. We choose the timer interval of a client module to be smaller compared to the timer interval of a control module. We do this to ensure a fast update of the data between the plant and the control module.

V-B Server

We show the implementation of a server that reads the sensor measurements and write actuator commands from and to a real-time simulator. We implement such a server as an infinite loop that waits for requests from a client. Listing 4 demonstrates a server for the CPS-RT-APC with server() representing the infinite loop. We pass the data for process variables applying the custom struct Args_sim_plant. We use pointers to pass the data of such a struct to both the server and real-time simulator. Such an implementation requires mutex-like principles to synchronize the communication between the server and the real-time simulator and we demonstrate this by locking and unlocking the code where server() manipulates the data in the shared struct.

Finally, such a client-server communication architecture enables cloud computing, storage, and remote monitoring of CPS-RT-APC systems. Fig. 4 illustrates how a client-server architecture for the CPS-RT-APC may enable the usage of remote computers for monitoring and cloud computing.

V-C Python Implementation

Python’s socket communication mechanism is similar to the socket functions applied in the code examples in Listings 3 and 4, i.e., sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) creates a socket server. We apply sock.bind(server_address) to bind the socket to the host-machine and we use sock.listen() to listen for clients. To connect a client to a server, we create a client socket with sock = socket.socket() and then apply sock.connect(ipaddress, port) [10].

VI Control computations

Fig. 3 outlines the fundamental tasks in a control module for the CPS-RT-APC and we demonstrate the implementation of these tasks in this section. However, it is also common to see logic tasks in the form of evaluating the state of actuators and sensors included in such a module. Listing 5 presents an example of such an implementation applying the callback function control(). This callback function requires the control function, the controller update function, the sampling time, and a connection to the shared data. We pass these as arguments to the callback function by using the struct Args_control.

VII Real-time Simulator

In this section, we demonstrate the implementation of a real-time simulator applying a POSIX interval timer. Listing 6 presents an example of such an implementation applying the callback function simulator(). This callback function requires a mathematical model of the plant, the states, inputs, and output dimensions, model parameters and control signals, and an ordinary differential equation (ODE) solver. We pass these as arguments to the callback function through the struct Args_sim_plant.

VIII User Interface

A user interface module enables an operator to view process data through real-time plotting of measurements and manipulated variables, and to influence the control system through manipulation of setpoints and control configurations. Such a user interface requires a connection to the shared data for exchanging information between the other modules as well as real-time interval timers for real-time plotting and an event-based mechanism for changing setpoints. Fig. 5 illustrates a graphical user interface for a CPS-RT-APC with real-time plotting of data from two sensors and two actuators and buttons and text-fields for manipulating setpoints, tuning parameters, and the operation mode.

IX Real-time simulation experiments

We test the CPS-RT-APC framework with a real-time simulator of the ODE $\dot{x}(t)=(1/\tau)(-x(t)+Ku(t))$ and with measurement equations $y(t_{k})=x(t_{k})$ and output equation $z(t)=x(t)$, where $t$ is time, $x(t)$ is the state, $u(t)$ is the manipulated variable, $y(t_{k})$ is the measurement, and $z(t)$ is the controlled variable. We choose the parameters $K=10.0$ and $\tau=10.0$ and the simulator timer interval $T_{s}^{p}$ = 0.2 s. We apply a zero-order hold parameterization of the manipulated variable $u(t)=u_{k}$ for $t_{k}^{p}\leq t\leq t_{k+1}^{p}$ with $t_{k+1}^{p}=t_{k}^{p}+T_{s}^{p}$ being the time increment for the simulator. We track a time-varying setpoint, $\bar{z}_{k}$, with the PI-control strategy $e_{k}=\bar{z}_{k}-y_{k}$, $P_{k}=K_{p}e_{k}$, $u_{k}=\bar{u}_{k}+P_{k}+I_{k}$, $I_{k+1}=I_{k}+({K_{p}T_{s}^{c}}/{\tau_{i}})e_{k}$, and $I_{0}=\bar{I}$ where $T_{s}^{c}$ is the interval time of the controller, and $K_{p}$, $\tau_{i}$, and $\bar{u}_{k}$ are the proportional gain, integral time constant, and chosen operating point for $u_{k}$, respectively. We choose the controller parameters $K_{p}=0.2$, $\tau_{i}=10.0$, $\bar{u}=0.0$, and the interval time $T_{s}^{c}=2.0$ s. We update a sequence of setpoints applying a real-time timer with interval $T_{s}^{\bar{z}}=150.0$ s. We apply a client module with interval $T_{s}^{cl}=0.5$ s and we start the timers simultaneously. We execute the control system and the simulator on the same host computer. Finally, we apply a PostgreSQL database system for storing the data for $y(t_{k}),u_{k}$ and $\bar{z}_{k}$. Fig. 6 presents histograms for the jitter $\Delta t_{k}^{i}=t_{k+1}^{i}-t_{k}^{i}$ where $t_{k+1}^{i}=t_{k}^{i}+T_{s}^{i}$ for $i\in\{p,c,cl\}$ as well as the process variables of the simulation experiment.

X Conclusion

In this paper, we demonstrate the key software principles and components applied in the implementation of cyber-physical systems for real-time advanced process control. These principles and components are shared data, timers and threads for concurrent periodic execution of tasks, and network communication between the control system and the process and between the control system and the internet. We demonstrate these principles and components with code examples in the C programming language for Linux operating systems. We apply these examples to a real-time simulation experiment of an ODE, where a PI-control strategy tracks a time-varying setpoint. The results show high timing precision and good setpoint tracking.