Deep Reinforcement Learning based Joint Active and Passive Beamforming Design for RIS-Assisted MISO Systems

TL;DR

This paper introduces a deep reinforcement learning approach using the soft actor-critic algorithm to jointly optimize active and passive beamforming in RIS-assisted mmWave MISO systems, improving performance over traditional methods.

Contribution

It develops a novel DRL-based joint beamforming design method for RIS-assisted systems, addressing computational complexity and environmental dynamics.

Findings

SAC-based DRL outperforms conventional optimization algorithms.

Joint design improves system performance in RIS-assisted mmWave MISO systems.

Proposed method effectively adapts to dynamic wireless environments.

Abstract

Owing to the unique advantages of low cost and controllability, reconfigurable intelligent surface (RIS) is a promising candidate to address the blockage issue in millimeter wave (mmWave) communication systems, consequently has captured widespread attention in recent years. However, the joint active beamforming and passive beamforming design is an arduous task due to the high computational complexity and the dynamic changes of wireless environment. In this paper, we consider a RIS-assisted multi-user multiple-input single-output (MU-MISO) mmWave system and aim to develop a deep reinforcement learning (DRL) based algorithm to jointly design active hybrid beamformer at the base station (BS) side and passive beamformer at the RIS side. By employing an advanced soft actor-critic (SAC) algorithm, we propose a maximum entropy based DRL algorithm, which can explore more stochastic policies than deterministic policy, to design active analog precoder and passive beamformer simultaneously. Then, the digital precoder is determined by minimum mean square error (MMSE) method. The experimental results demonstrate that our proposed SAC algorithm can achieve better performance compared with conventional optimization algorithm and DRL algorithm.