Scalable machine learning-based approaches for energy saving in densely deployed Open RAN

TL;DR

This paper introduces scalable deep reinforcement learning methods for energy-efficient management of densely deployed Open RAN base stations, demonstrating significant energy savings and faster convergence.

Contribution

It proposes novel federated and centralized DRL solutions for controlling radio unit sleep modes, enhancing scalability and energy efficiency in Open RAN networks.

Findings

Federated DRL achieves up to 43.75% faster convergence.

Energy savings exceed 50% compared to baseline methods.

The proposed methods maintain quality of service and improve robustness.

Abstract

Densely deployed base stations are responsible for the majority of the energy consumed in Radio access network (RAN). While these deployments are crucial to deliver the required data rate in busy hours of the day, the network can save energy by switching some of them to sleep mode and maintain the coverage and quality of service with the other ones. Benefiting from the flexibility provided by the Open RAN in embedding machine learning (ML) in network operations, in this work we propose Deep Reinforcement Learning (DRL)-based energy saving solutions. Firstly we propose 3 different DRL-based methods in the form of xApps which control the Active/Sleep mode of up to 6 radio units (RUs) from Near Real time RAN Intelligent Controller (RIC). We also propose a further scalable federated DRL-based solution with an aggregator as an rApp in None Real time RIC and local agents as xApps. Our simulation results present the convergence of the proposed methods. We also compare the performance of our federated DRL across three layouts spanning 6--24 RUs and 500--1000\,m regions, including a composite multi-region scenario. The results show that our proposed federated TD3 algorithm achieves up to 43.75\% faster convergence, more than 50\% network energy saving and 37. 4\% lower training energy versus centralized baselines, while maintaining the quality of service and improving the robustness of the policy.