Intent-driven Intelligent Control and Orchestration in O-RAN Via Hierarchical Reinforcement Learning

TL;DR

This paper introduces a hierarchical reinforcement learning approach for intent-driven control and orchestration of rApps and xApps in O-RAN, improving network performance and efficiency in multi-vendor environments.

Contribution

It presents a novel bi-level HRL architecture for orchestrating network applications based on operator-defined KPIs, enhancing adaptability and performance.

Findings

7.5% increase in system throughput over single xApp baseline

17.3% improvement in energy efficiency compared to non-ML algorithms

Achieves better KPI optimization in complex multi-vendor O-RAN environments

Abstract

rApps and xApps need to be controlled and orchestrated well in the open radio access network (O-RAN) so that they can deliver a guaranteed network performance in a complex multi-vendor environment. This paper proposes a novel intent-driven intelligent control and orchestration scheme based on hierarchical reinforcement learning (HRL). The proposed scheme can orchestrate multiple rApps or xApps according to the operator's intent of optimizing certain key performance indicators (KPIs), such as throughput, energy efficiency, and latency. Specifically, we propose a bi-level architecture with a meta-controller and a controller. The meta-controller provides the target performance in terms of KPIs, while the controller performs xApp orchestration at the lower level. Our simulation results show that the proposed HRL-based intent-driven xApp orchestration mechanism achieves 7.5% and 21.4% increase in average system throughput with respect to two baselines, i.e., a single xApp baseline and a non-machine learning-based algorithm, respectively. Similarly, 17.3% and 37.9% increase in energy efficiency are observed in comparison to the same baselines.