A DRL Approach for RIS-Assisted Full-Duplex UL and DL Transmission: Beamforming, Phase Shift and Power Optimization

TL;DR

This paper introduces a two-stage deep reinforcement learning framework for RIS-assisted full-duplex transmission that optimizes beamforming, phase shifts, and power with minimal signaling, achieving high rates without requiring channel state information.

Contribution

The paper presents a novel two-stage DRL approach that reduces signaling overhead and eliminates the need for CSI, enabling efficient RIS-assisted full-duplex communication.

Findings

Quantized RIS phase shifts reduce signaling by 32 times.

The proposed method achieves 7.1% and 22.28% higher UL and DL rates.

Faster convergence compared to continuous phase shift methods.

Abstract

We propose a deep reinforcement learning (DRL) approach for a full-duplex (FD) transmission that predicts the phase shifts of the reconfigurable intelligent surface (RIS), base station (BS) active beamformers, and the transmit powers to maximize the weighted sum rate of uplink and downlink users. Existing methods require channel state information (CSI) and residual self-interference (SI) knowledge to calculate exact active beamformers or the DRL rewards, which typically fail without CSI or residual SI. Especially for time-varying channels, estimating and signaling CSI to the DRL agent is required at each time step and is costly. We propose a two-stage DRL framework with minimal signaling overhead to address this. The first stage uses the least squares method to initiate learning by partially canceling the residual SI. The second stage uses DRL to achieve performance comparable to existing CSI-based methods without requiring the CSI or the exact residual SI. Further, the proposed DRL framework for quantized RIS phase shifts reduces the signaling from BS to the RISs using $32$ times fewer bits than the continuous version. The quantized methods reduce action space, resulting in faster convergence and $7.1\%$ and $22.28\%$ better UL and DL rates, respectively than the continuous method.