Hierarchical Reinforcement Learning Based Traffic Steering in Multi-RAT 5G Deployments

TL;DR

This paper introduces a hierarchical reinforcement learning approach for traffic steering in 5G multi-RAT environments, improving throughput and reducing delay by intelligently balancing load and QoS requirements.

Contribution

It proposes a novel load-aware HRL algorithm with a bi-level architecture for traffic management in 5G, outperforming existing DQN and heuristic methods.

Findings

HRL achieves 8.49% higher throughput than DQN.

HRL reduces network delay by up to 39.13%.

Bi-level HRL outperforms baseline methods in simulations.

Abstract

In 5G non-standalone mode, an intelligent traffic steering mechanism can vastly aid in ensuring smooth user experience by selecting the best radio access technology (RAT) from a multi-RAT environment for a specific traffic flow. In this paper, we propose a novel load-aware traffic steering algorithm based on hierarchical reinforcement learning (HRL) while satisfying diverse QoS requirements of different traffic types. HRL can significantly increase system performance using a bi-level architecture having a meta-controller and a controller. In our proposed method, the meta-controller provides an appropriate threshold for load balancing, while the controller performs traffic admission to an appropriate RAT in the lower level. Simulation results show that HRL outperforms a Deep Q-Learning (DQN) and a threshold-based heuristic baseline with 8.49%, 12.52% higher average system throughput and 27.74%, 39.13% lower network delay, respectively.