Intelligent Reflecting Surface Configurations for Smart Radio Using Deep Reinforcement Learning

TL;DR

This paper introduces a novel model-free IRS control scheme using deep reinforcement learning and extremum seeking control, enabling real-time adaptive phase control without relying on sub-channel CSI in complex wireless environments.

Contribution

It proposes a joint DRL and ESC framework for IRS control that is independent of channel state information, improving adaptability and practical implementation in smart radio systems.

Findings

The joint DRL and ESC scheme outperforms traditional methods in simulations.

The approach effectively adapts to different channel dynamics.

It demonstrates real-time IRS control without sub-channel CSI.

Abstract

Intelligent reflecting surface (IRS) is envisioned to change the paradigm of wireless communications from "adapting to wireless channels" to "changing wireless channels". However, current IRS configuration schemes, consisting of sub-channel estimation and passive beamforming in sequence, conform to the conventional model-based design philosophies and are difficult to be realized practically in the complex radio environment. To create the smart radio environment, we propose a model-free design of IRS control that is independent of the sub-channel channel state information (CSI) and requires the minimum interaction between IRS and the wireless communication system. We firstly model the control of IRS as a Markov decision process (MDP) and apply deep reinforcement learning (DRL) to perform real-time coarse phase control of IRS. Then, we apply extremum seeking control (ESC) as the fine phase control of IRS. Finally, by updating the frame structure, we integrate DRL and ESC in the model-free control of IRS to improve its adaptivity to different channel dynamics. Numerical results show the superiority of our proposed joint DRL and ESC scheme and verify its effectiveness in model-free IRS control without sub-channel CSI.