Foundation Model-Aided Hierarchical Deep Reinforcement Learning for Blockage-Aware Link in RIS-Assisted Networks

TL;DR

This paper introduces a foundation model-aided hierarchical deep reinforcement learning framework that enhances spectral efficiency in RIS-assisted wireless networks by optimizing beamforming and phase shifts under dynamic conditions.

Contribution

It proposes a novel FM-HDRL framework that combines large wireless models with hierarchical reinforcement learning for joint optimization in RIS-assisted networks.

Findings

Achieves 7.82% higher spectral efficiency than FM-DRL.

Improves about 48.66% spectral efficiency over beam sweeping.

Demonstrates faster convergence and better scalability.

Abstract

Reconfigurable intelligent surface (RIS) technology has the potential to significantly enhance the spectral efficiency (SE) of 6G wireless networks. However, practical deployment remains constrained by challenges in accurate channel estimation and control optimization under dynamic conditions. This paper presents a foundation model-aided hierarchical deep reinforcement learning (FM-HDRL) framework designed for joint beamforming and phase-shift optimization in RIS-assisted wireless networks. To implement this, we first fine-tune a pre-trained large wireless model (LWM) to translate raw channel data into low-dimensional, context-aware channel state information (CSI) embeddings. Next, these embeddings are combined with user location information and blockage status to select the optimal communication path. The resulting features are then fed into an HDRL model, assumed to be implemented at a centralized controller, which jointly optimizes the base station (BS) beamforming vectors and the RIS phase-shift configurations to maximize SE. Simulation results demonstrate that the proposed FM-HDRL framework consistently outperforms baseline methods in terms of convergence speed, spectral efficiency, and scalability. According to the simulation results, our proposed method improves 7.82% SE compared to the FM-aided deep reinforcement learning (FM-DRL) approach and a substantial enhancement of about 48.66% relative to the beam sweeping approach.