A Learning Approach for Joint Design of Event-triggered Control and Power-Efficient Resource Allocation

TL;DR

This paper introduces a hierarchical reinforcement learning method for joint event-triggered control and energy-efficient resource allocation in 5G industrial systems, reducing actuator updates and power use.

Contribution

It presents a novel model-free RL framework that simultaneously learns control and resource policies with stability guarantees for ICPSs.

Findings

Reduces actuator input updates significantly

Decreases downlink power consumption

Maintains system stability

Abstract

In emerging Industrial Cyber-Physical Systems (ICPSs), the joint design of communication and control sub-systems is essential, as these sub-systems are interconnected. In this paper, we study the joint design problem of an event-triggered control and an energy-efficient resource allocation in a fifth generation (5G) wireless network. We formally state the problem as a multi-objective optimization one, aiming to minimize the number of updates on the actuators' input and the power consumption in the downlink transmission. To address the problem, we propose a model-free hierarchical reinforcement learning approach \textcolor{blue}{with uniformly ultimate boundedness stability guarantee} that learns four policies simultaneously. These policies contain an update time policy on the actuators' input, a control policy, and energy-efficient sub-carrier and power allocation policies. Our simulation results show that the proposed approach can properly control a simulated ICPS and significantly decrease the number of updates on the actuators' input as well as the downlink power consumption.