Stability of multiplexed NCS based on an epsilon-greedy algorithm for communication selection

TL;DR

This paper introduces an epsilon-greedy algorithm for communication selection in multiplexed Networked Control Systems, ensuring stability and optimizing control performance through deep reinforcement learning.

Contribution

It proposes a novel epsilon-greedy approach for communication scheduling in NCS that guarantees stability and leverages deep Q-learning for optimal control.

Findings

The epsilon-greedy algorithm ensures Mean Square Stability in multiplexed NCS.

The deep Q network optimizes communication sequences to minimize quadratic cost.

Numerical results show improved performance over round-robin and random schemes.

Abstract

In this letter, we study a Networked Control System (NCS) with multiplexed communication and Bernoulli packet drops. Multiplexed communication refers to the constraint that transmission of a control signal and an observation signal cannot occur simultaneously due to the limited bandwidth. First, we propose an epsilon-greedy algorithm for the selection of the communication sequence that also ensures Mean Square Stability (MSS). We formulate the system as a Markovian Jump Linear System (MJLS) and provide the necessary conditions for MSS in terms of Linear Matrix Inequalities (LMIs) that need to be satisfied for three corner cases. We prove that the system is MSS for any convex combination of these three corner cases. Furthermore, we propose to use the epsilon-greedy algorithm with the epsilon that satisfies MSS conditions for training a Deep Q Network (DQN). The DQN is used to obtain an optimal communication sequence that minimizes a quadratic cost. We validate our approach with a numerical example that shows the efficacy of our method in comparison to the round-robin and a random scheme.