Task Specific Sharpness Aware O-RAN Resource Management using Multi Agent Reinforcement Learning

TL;DR

This paper presents a novel multi-agent reinforcement learning method that combines Sharpness-Aware Minimization with adaptive regularization to improve resource management in O-RAN networks, enhancing robustness and efficiency.

Contribution

It introduces an adaptive, TD-error driven SAM mechanism within a multi-agent RL framework, improving generalization and training stability for O-RAN resource management.

Findings

Up to 22% improvement in resource allocation efficiency

Enhanced QoS satisfaction across diverse O-RAN slices

Reduced training overhead and increased stability

Abstract

Next-generation networks utilize the Open Radio Access Network (O-RAN) architecture to enable dynamic resource management, facilitated by the RAN Intelligent Controller (RIC). While deep reinforcement learning (DRL) models show promise in optimizing network resources, they often struggle with robustness and generalizability in dynamic environments. This paper introduces a novel resource management approach that enhances the Soft Actor Critic (SAC) algorithm with Sharpness-Aware Minimization (SAM) in a distributed Multi-Agent RL (MARL) framework. Our method introduces an adaptive and selective SAM mechanism, where regularization is explicitly driven by temporal-difference (TD)-error variance, ensuring that only agents facing high environmental complexity are regularized. This targeted strategy reduces unnecessary overhead, improves training stability, and enhances generalization without sacrificing learning efficiency. We further incorporate a dynamic $\rho$ scheduling scheme to refine the exploration-exploitation trade-off across agents. Experimental results show our method significantly outperforms conventional DRL approaches, yielding up to a $22\%$ improvement in resource allocation efficiency and ensuring superior QoS satisfaction across diverse O-RAN slices.