Meta Hierarchical Reinforcement Learning for Scalable Resource Management in O-RAN

TL;DR

This paper introduces a Meta Hierarchical Reinforcement Learning framework for scalable, adaptive resource management in O-RAN, combining hierarchical control and meta learning to enhance network efficiency and adaptability under dynamic conditions.

Contribution

The paper presents a novel Meta-HRL framework inspired by MAML that jointly optimizes resource allocation and network slicing in O-RAN, with theoretical guarantees and improved simulation performance.

Findings

19.8% improvement in network management efficiency

Faster adaptation by up to 40%

Robust scalability with consistent performance

Abstract

The increasing complexity of modern applications demands wireless networks capable of real time adaptability and efficient resource management. The Open Radio Access Network (O-RAN) architecture, with its RAN Intelligent Controller (RIC) modules, has emerged as a pivotal solution for dynamic resource management and network slicing. While artificial intelligence (AI) driven methods have shown promise, most approaches struggle to maintain performance under unpredictable and highly dynamic conditions. This paper proposes an adaptive Meta Hierarchical Reinforcement Learning (Meta-HRL) framework, inspired by Model Agnostic Meta Learning (MAML), to jointly optimize resource allocation and network slicing in O-RAN. The framework integrates hierarchical control with meta learning to enable both global and local adaptation: the high-level controller allocates resources across slices, while low level agents perform intra slice scheduling. The adaptive meta-update mechanism weights tasks by temporal difference error variance, improving stability and prioritizing complex network scenarios. Theoretical analysis establishes sublinear convergence and regret guarantees for the two-level learning process. Simulation results demonstrate a 19.8% improvement in network management efficiency compared with baseline RL and meta-RL approaches, along with faster adaptation and higher QoS satisfaction across eMBB, URLLC, and mMTC slices. Additional ablation and scalability studies confirm the method's robustness, achieving up to 40% faster adaptation and consistent fairness, latency, and throughput performance as network scale increases.