DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling

TL;DR

DeepCAS employs deep reinforcement learning to develop control-aware scheduling for networked control systems, optimizing communication schedules to minimize control loss in large-scale cyber-physical systems.

Contribution

It introduces a novel deep reinforcement learning algorithm that adapts scheduling based on subsystem controllers to improve control performance.

Findings

DeepCAS outperforms periodic scheduling in reducing control loss.

The algorithm effectively adapts to different subsystems and controllers.

Empirical results demonstrate improved scheduling efficiency.

Abstract

We consider networked control systems consisting of multiple independent controlled subsystems, operating over a shared communication network. Such systems are ubiquitous in cyber-physical systems, Internet of Things, and large-scale industrial systems. In many large-scale settings, the size of the communication network is smaller than the size of the system. In consequence, scheduling issues arise. The main contribution of this paper is to develop a deep reinforcement learning-based \emph{control-aware} scheduling (\textsc{DeepCAS}) algorithm to tackle these issues. We use the following (optimal) design strategy: First, we synthesize an optimal controller for each subsystem; next, we design a learning algorithm that adapts to the chosen subsystems (plants) and controllers. As a consequence of this adaptation, our algorithm finds a schedule that minimizes the \emph{control loss}. We present empirical results to show that \textsc{DeepCAS} finds schedules with better performance than periodic ones.