Model-Free Design of Control Systems over Wireless Fading Channels

TL;DR

This paper introduces a model-free reinforcement learning approach for designing control and resource allocation policies in wireless control systems, addressing challenges of infinite dimensionality and system constraints.

Contribution

It proposes a novel model-free algorithm for co-designing control-aware resource allocation and control policies in wireless networks, ensuring near-optimal performance.

Findings

Near-optimal control system performance demonstrated

Strong performance of learned policies over baselines

Effective handling of wireless fading and system constraints

Abstract

Wireless control systems replace traditional wired communication with wireless networks to exchange information between actuators, plants and sensors in a control system. The noise in wireless channels renders ideal control policies suboptimal, and their performance is moreover directly dependent on the way in which wireless resources are allocated between control loops. Proper design of the control policy and the resource allocation policy based on both plant states and wireless fading states is then critical to achieve good performance. The resulting problem of co-designing control-aware resource allocation policies and communication-aware controllers, however, is challenging due to its infinite dimensionality, existence of system constraints and need for explicit knowledge of the plants and wireless network models. To overcome those challenges, we rely on constrained reinforcement learning algorithms to propose a model-free approach to the design of wireless control systems. We demonstrate the near optimality of control system performance and stability using near-universal policy parametrizations and present a practical model-free algorithm to learn the co-design policy. Numerical experiments show the strong performance of learned policies over baseline solutions.