Multi-Agent Deep Reinforcement Learning for Energy Efficient Multi-Hop STAR-RIS-Assisted Transmissions

TL;DR

This paper introduces a multi-hop STAR-RIS architecture and a multi-agent deep reinforcement learning algorithm to enhance energy efficiency and coverage in wireless communications, outperforming existing methods.

Contribution

It proposes a novel multi-hop STAR-RIS architecture combined with MAGAR, a multi-agent DRL algorithm, to optimize beamforming for energy efficiency in wireless networks.

Findings

MAGAR significantly outperforms benchmark algorithms.

Multi-hop STAR-RIS achieves higher energy efficiency than mode switching and conventional RIS.

The on-off state of STAR-RIS elements impacts energy efficiency.

Abstract

Simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) provides a promising way to expand coverage in wireless communications. However, limitation of single STAR-RIS inspire us to integrate the concept of multi-hop transmissions, as focused on RIS in existing research. Therefore, we propose the novel architecture of multi-hop STAR-RISs to achieve a wider range of full-plane service coverage. In this paper, we intend to solve active beamforming of the base station and passive beamforming of STAR-RISs, aiming for maximizing the energy efficiency constrained by hardware limitation of STAR-RISs. Furthermore, we investigate the impact of the on-off state of STAR-RIS elements on energy efficiency. To tackle the complex problem, a Multi-Agent Global and locAl deep Reinforcement learning (MAGAR) algorithm is designed. The global agent elevates the collaboration among local agents, which focus on individual learning. In numerical results, we observe the significant improvement of MAGAR compared to the other benchmarks, including Q-learning, multi-agent deep Q network (DQN) with golbal reward, and multi-agent DQN with local rewards. Moreover, the proposed architecture of multi-hop STAR-RISs achieves the highest energy efficiency compared to mode switching based STAR-RISs, conventional RISs and deployment without RISs or STAR-RISs.