DRL-based Joint Beamforming and BS-RIS-UE Association Design for RIS-Assisted mmWave Networks

TL;DR

This paper introduces a DRL-based method to jointly optimize beamforming and association in RIS-assisted mmWave networks, significantly improving system sum-rate by intelligently managing RIS, BS, and UE interactions.

Contribution

It proposes a novel SAC-DRL algorithm for joint beamforming and association, effectively handling the non-convex problem with less prior information and avoiding local optima.

Findings

Achieves significant sum-rate improvements over benchmarks.

Effectively learns optimal association and beamforming policies.

Demonstrates robustness and efficiency in simulation results.

Abstract

Reconfigurable intelligent surface (RIS) is considered as an extraordinarily promising technology to solve the blockage problem of millimeter wave (mmWave) communications owing to its capable of establishing a reconfigurable wireless propagation. In this paper, we focus on a RIS-assisted mmWave communication network consisting of multiple base stations (BSs) serving a set of user equipments (UEs). Considering the BS-RIS-UE association problem which determines that the RIS should assist which BS and UEs, we joint optimize BS-RIS-UE association and passive beamforming at RIS to maximize the sum-rate of the system. To solve this intractable non-convex problem, we propose a soft actor-critic (SAC) deep reinforcement learning (DRL)-based joint beamforming and BS-RIS-UE association design algorithm, which can learn the best policy by interacting with the environment using less prior information and avoid falling into the local optimal solution by incorporating with the maximization of policy information entropy. The simulation results demonstrate that the proposed SAC-DRL algorithm can achieve significant performance gains compared with benchmark schemes.