Enabling Deep Reinforcement Learning Research for Energy Saving in Open RAN

TL;DR

This paper presents a framework that leverages deep reinforcement learning to optimize energy efficiency in 5G Open RAN systems, using realistic simulation environments and open-source tools.

Contribution

It introduces an open-source framework integrating ns-O-RAN and Gymnasium for training and evaluating DRL agents in energy management of 5G networks.

Findings

DRL agents can effectively control cell activation for energy savings

The framework supports realistic 5G scenarios with mobility and protocol compliance

Open-source tools facilitate research and reproducibility in energy-efficient RAN design

Abstract

The growing performance demands and higher deployment densities of next-generation wireless systems emphasize the importance of adopting strategies to manage the energy efficiency of mobile networks. In this demo, we showcase a framework that enables research on Deep Reinforcement Learning (DRL) techniques for improving the energy efficiency of intelligent and programmable Open Radio Access Network (RAN) systems. Using the open-source simulator ns-O-RAN and the reinforcement learning environment Gymnasium, the framework enables to train and evaluate DRL agents that dynamically control the activation and deactivation of cells in a 5G network. We show how to collect data for training and evaluate the impact of DRL on energy efficiency in a realistic 5G network scenario, including users' mobility and handovers, a full protocol stack, and 3rd Generation Partnership Project (3GPP)-compliant channel models. The tool will be open-sourced and a tutorial for energy efficiency testing in ns-O-RAN.