Radio Resource and Beam Management in 5G mmWave Using Clustering and Deep Reinforcement Learning

TL;DR

This paper introduces a novel online clustering and deep reinforcement learning-based resource management scheme for 5G mmWave networks, improving latency, reliability, and throughput for diverse user types amid mobility.

Contribution

It proposes a QoS-aware clustering and resource allocation method combining DBSCAN and LSTM-based DRL, specifically addressing user mobility and traffic variability in 5G mmWave networks.

Findings

Outperforms baseline in latency and reliability for URLLC users

Enhances data rates for eMBB users

Effectively manages dynamic user mobility and traffic patterns

Abstract

To optimally cover users in millimeter-Wave (mmWave) networks, clustering is needed to identify the number and direction of beams. The mobility of users motivates the need for an online clustering scheme to maintain up-to-date beams towards those clusters. Furthermore, mobility of users leads to varying patterns of clusters (i.e., users move from the coverage of one beam to another), causing dynamic traffic load per beam. As such, efficient radio resource allocation and beam management is needed to address the dynamicity that arises from mobility of users and their traffic. In this paper, we consider the coexistence of Ultra-Reliable Low-Latency Communication (URLLC) and enhanced Mobile BroadBand (eMBB) users in 5G mmWave networks and propose a Quality-of-Service (QoS) aware clustering and resource allocation scheme. Specifically, Density-Based Spatial Clustering of Applications with Noise (DBSCAN) is used for online clustering of users and the selection of the number of beams. In addition, Long Short Term Memory (LSTM)-based Deep Reinforcement Learning (DRL) scheme is used for resource block allocation. The performance of the proposed scheme is compared to a baseline that uses K-means and priority-based proportional fairness for clustering and resource allocation, respectively. Our simulation results show that the proposed scheme outperforms the baseline algorithm in terms of latency, reliability, and rate of URLLC users as well as rate of eMBB users.