Hybrid Deep Reinforcement Learning for Joint Resource Allocation in Multi-Active RIS-Aided Uplink Communications

TL;DR

This paper introduces a hybrid deep reinforcement learning framework for resource allocation in multi-user uplink systems aided by multiple active RISs, aiming to maximize the minimum user rate efficiently.

Contribution

It proposes a novel combination of DRL algorithms with a closed-form beamforming solution for joint optimization in RIS-assisted uplink communications.

Findings

SAC outperforms DDPG and TD3 in convergence speed and computational efficiency.

Closed-form beamforming significantly improves the minimum user rate.

The hybrid DRL approach effectively manages high-dimensional resource allocation.

Abstract

Active Reconfigurable Intelligent Surfaces (RIS) are a promising technology for 6G wireless networks. This paper investigates a novel hybrid deep reinforcement learning (DRL) framework for resource allocation in a multi-user uplink system assisted by multiple active RISs. The objective is to maximize the minimum user rate by jointly optimizing user transmit powers, active RIS configurations, and base station (BS) beamforming. We derive a closed-form solution for optimal beamforming and employ DRL algorithms: Soft actor-critic (SAC), deep deterministic policy gradient (DDPG), and twin delayed DDPG (TD3) to solve the high-dimensional, non-convex power and RIS optimization problem. Simulation results demonstrate that SAC achieves superior performance with high learning rate leading to faster convergence and lower computational cost compared to DDPG and TD3. Furthermore, the closed-form of optimally beamforming enhances the minimum rate effectively.