Offline Reinforcement Learning for Mobility Robustness Optimization

TL;DR

This paper explores offline reinforcement learning techniques to optimize mobility robustness in cellular networks, demonstrating improved performance and flexibility over traditional rule-based methods using real network data.

Contribution

It introduces the application of offline RL methods, Decision Transformers and Conservative Q-Learning, to optimize cell offset tuning in mobility management, outperforming rule-based approaches.

Findings

Offline RL methods outperform rule-based MRO by up to 7%.

Offline RL can be trained for diverse objectives using the same dataset.

The approach demonstrates improved robustness and operational flexibility.

Abstract

In this work we revisit the Mobility Robustness Optimisation (MRO) algorithm and study the possibility of learning the optimal Cell Individual Offset tuning using offline Reinforcement Learning. Such methods make use of collected offline datasets to learn the optimal policy, without further exploration. We adapt and apply a sequence-based method called Decision Transformers as well as a value-based method called Conservative Q-Learning to learn the optimal policy for the same target reward as the vanilla rule-based MRO. The same input features related to failures, ping-pongs, and other handover issues are used. Evaluation for realistic New Radio networks with 3500 MHz carrier frequency on a traffic mix including diverse user service types and a specific tunable cell-pair shows that offline-RL methods outperform rule-based MRO, offering up to 7% improvement. Furthermore, offline-RL can be trained for diverse objective functions using the same available dataset, thus offering operational flexibility compared to rule-based methods.