Deep reinforcement learning for RAN optimization and control

TL;DR

This paper demonstrates the successful application of deep reinforcement learning to optimize radio access network performance in a real-world lab environment, addressing the complexity and variability of RAN configurations.

Contribution

It introduces a deep RL-based controller for RAN optimization that operates without domain-specific assumptions and is validated in a real lab setup.

Findings

Deep RL improves RAN KPIs in real environment

The control agent adapts to network variability

Demonstrates feasibility of RL in live RAN management

Abstract

Due to the high variability of the traffic in the radio access network (RAN), fixed network configurations are not flexible enough to achieve optimal performance. Our vendors provide several settings of the eNodeB to optimize the RAN performance, such as media access control scheduler, loading balance, etc. But the detailed mechanisms of the eNodeB configurations are usually very complicated and not disclosed, not to mention the large key performance indicators (KPIs) space needed to be considered. These make constructing a simulator, offline tuning, or rule-based solutions difficult. We aim to build an intelligent controller without strong assumption or domain knowledge about the RAN and can run 24/7 without supervision. To achieve this goal, we first build a closed-loop control testbed RAN in a lab environment with one eNodeB provided by one of the largest wireless vendors and four smartphones. Next, we build a double Q network agent trained with the live feedback of the key performance indicators from the RAN. Our work proved the effectiveness of applying deep reinforcement learning to improve network performance in a real RAN network environment.