Joint Sensing and Communications for Deep Reinforcement Learning-based Beam Management in 6G

TL;DR

This paper introduces a vision-aided joint sensing and communication approach using deep reinforcement learning for beam management in 6G mmWave networks, effectively reducing localization errors and improving network performance.

Contribution

It proposes a novel method combining satellite image features, UK-medoids clustering, and DRL for enhanced beam management under user location uncertainty in 6G networks.

Findings

Significantly reduces localization error with vision-aided method.

Achieves 17.2% higher throughput and 7.7% lower delay compared to UK-DRL.

More than doubles throughput and reduces delay by 55.8% compared to K-DRL.

Abstract

User location is a piece of critical information for network management and control. However, location uncertainty is unavoidable in certain settings leading to localization errors. In this paper, we consider the user location uncertainty in the mmWave networks, and investigate joint vision-aided sensing and communications using deep reinforcement learning-based beam management for future 6G networks. In particular, we first extract pixel characteristic-based features from satellite images to improve localization accuracy. Then we propose a UK-medoids based method for user clustering with location uncertainty, and the clustering results are consequently used for the beam management. Finally, we apply the DRL algorithm for intra-beam radio resource allocation. The simulations first show that our proposed vision-aided method can substantially reduce the localization error. The proposed UK-medoids and DRL based scheme (UKM-DRL) is compared with two other schemes: K-means based clustering and DRL based resource allocation (K-DRL) and UK-means based clustering and DRL based resource allocation (UK-DRL). The proposed method has 17.2% higher throughput and 7.7% lower delay than UK-DRL, and more than doubled throughput and 55.8% lower delay than K-DRL.