Percentile-Based Deep Reinforcement Learning and Reward Based Personalization For Delay Aware RAN Slicing in O-RAN

TL;DR

This paper introduces a percentile-based deep reinforcement learning approach for delay-aware RAN slicing in O-RAN, optimizing resource allocation for multiple MVNOs and enhancing delay performance through personalized model sharing.

Contribution

It proposes a novel percentile-based DRL method for delay constraints and a reward-based personalization technique for multi-MVNO model sharing in RAN slicing.

Findings

Achieves 38% reduction in average delay compared to baselines.

Demonstrates superiority of percentile-based DRL over average delay optimization.

Introduces a personalized model sharing method outperforming federated averaging.

Abstract

In this paper, we tackle the challenge of radio access network (RAN) slicing within an open RAN (O-RAN) architecture. Our focus centers on a network that includes multiple mobile virtual network operators (MVNOs) competing for physical resource blocks (PRBs) with the goal of meeting probabilistic delay upper bound constraints for their clients while minimizing PRB utilization. Initially, we derive a reward function based on the law of large numbers (LLN), then implement practical modifications to adapt it for real-world experimental scenarios. We then propose our solution, the Percentile-based Delay-Aware Deep Reinforcement Learning (PDA-DRL), which demonstrates its superiority over several baselines, including DRL models optimized for average delay constraints, by achieving a 38\% reduction in resultant average delay. Furthermore, we delve into the issue of model weight sharing among multiple MVNOs to develop a robust personalized model. We introduce a reward-based personalization method where each agent prioritizes other agents' model weights based on their performance. This technique surpasses traditional aggregation methods, such as federated averaging, and strategies reliant on traffic patterns and model weight distance similarities.