Machine Learning-enabled Traffic Steering in O-RAN: A Case Study on Hierarchical Learning Approach

TL;DR

This paper surveys ML-based traffic steering in O-RAN and introduces a hierarchical deep reinforcement learning framework that improves performance by decomposing the problem into high-level and low-level decision-making layers.

Contribution

It proposes a novel hierarchical deep Q-learning framework for traffic steering in O-RAN, addressing limitations of single-layer architectures and demonstrating significant performance gains.

Findings

Hierarchical learning outperforms baseline algorithms in traffic steering.

The h-DQN framework effectively decomposes decision-making into high-level and low-level tasks.

Significant performance improvements are achieved through the proposed approach.

Abstract

Traffic Steering is a crucial technology for wireless networks, and multiple efforts have been put into developing efficient Machine Learning (ML)-enabled traffic steering schemes for Open Radio Access Networks (O-RAN). Given the swift emergence of novel ML techniques, conducting a timely survey that comprehensively examines the ML-based traffic steering schemes in O-RAN is critical. In this article, we provide such a survey along with a case study of hierarchical learning-enabled traffic steering in O-RAN. In particular, we first introduce the background of traffic steering in O-RAN and overview relevant state-of-the-art ML techniques and their applications. Then, we analyze the compatibility of the hierarchical learning framework in O-RAN and further propose a Hierarchical Deep-Q-Learning (h-DQN) framework for traffic steering. Compared to existing works, which focus on single-layer architecture with standalone agents, h-DQN decomposes the traffic steering problem into a bi-level architecture with hierarchical intelligence. The meta-controller makes long-term and high-level policies, while the controller executes instant traffic steering actions under high-level policies. Finally, the case study shows that the hierarchical learning approach can provide significant performance improvements over the baseline algorithms.