Federated Neuroevolution O-RAN: Enhancing the Robustness of Deep Reinforcement Learning xApps

TL;DR

This paper proposes a federated neuroevolution-enhanced deep reinforcement learning framework for O-RAN, improving the robustness of RAN intelligent controllers while maintaining operational efficiency.

Contribution

It introduces a novel NE-based optimizer xApp deployed alongside DRL xApps in O-RAN, enhancing exploration and robustness without disrupting RAN operations.

Findings

Improved robustness of xApps demonstrated in numerical results.

Effective balancing of computational load achieved.

Parallel deployment maintains RAN operation stability.

Abstract

The open radio access network (O-RAN) architecture introduces RAN intelligent controllers (RICs) to facilitate the management and optimization of the disaggregated RAN. Reinforcement learning (RL) and its advanced form, deep RL (DRL), are increasingly employed for designing intelligent controllers, or xApps, to be deployed in the near-real time (near-RT) RIC. These models often encounter local optima, which raise concerns about their reliability for RAN intelligent control. We therefore introduce Federated O-RAN enabled Neuroevolution (NE)-enhanced DRL (F-ONRL) that deploys an NE-based optimizer xApp in parallel to the RAN controller xApps. This NE-DRL xApp framework enables effective exploration and exploitation in the near-RT RIC without disrupting RAN operations. We implement the NE xApp along with a DRL xApp and deploy them on Open AI Cellular (OAIC) platform and present numerical results that demonstrate the improved robustness of xApps while effectively balancing the additional computational load.