Transfer Reinforcement Learning for 5G-NR mm-Wave Networks

TL;DR

This paper explores machine learning algorithms, including transfer Q-learning, to optimize user association and beam selection in 5G mm-Wave networks, significantly improving capacity and convergence speed amid interference challenges.

Contribution

It introduces transfer Q-learning for interference mitigation in 5G mm-Wave networks, demonstrating notable performance improvements over existing algorithms.

Findings

TQL and Q-learning outperform BSDC under mobility with 12% rate increase.

Q-learning and BSDC excel in stationary scenarios.

TQL achieves 29% faster convergence than Q-learning.

Abstract

In this paper, we aim at interference mitigation in 5G millimeter-Wave (mm-Wave) communications by employing beamforming and Non-Orthogonal Multiple Access (NOMA) techniques with the aim of improving network's aggregate rate. Despite the potential capacity gains of mm-Wave and NOMA, many technical challenges might hinder that performance gain. In particular, the performance of Successive Interference Cancellation (SIC) diminishes rapidly as the number of users increases per beam, which leads to higher intra-beam interference. Furthermore, intersection regions between adjacent cells give rise to inter-beam inter-cell interference. To mitigate both interference levels, optimal selection of the number of beams in addition to best allocation of users to those beams is essential. In this paper, we address the problem of joint user-cell association and selection of number of beams for the purpose of maximizing the aggregate network capacity. We propose three machine learning-based algorithms; transfer Q-learning (TQL), Q-learning, and Best SINR association with Density-based Spatial Clustering of Applications with Noise (BSDC) algorithms and compare their performance under different scenarios. Under mobility, TQL and Q-learning demonstrate 12% rate improvement over BSDC at the highest offered traffic load. For stationary scenarios, Q-learning and BSDC outperform TQL, however TQL achieves about 29% convergence speedup compared to Q-learning.