Generative Resource Allocation for 6G O-RAN with Diffusion Policies

TL;DR

This paper proposes Diffusion Q-Learning, a novel policy framework using diffusion models for dynamic resource allocation in 6G O-RAN, outperforming traditional reinforcement learning methods in complex network scenarios.

Contribution

Introduces Diffusion Q-Learning, a new approach that models resource allocation policies as diffusion processes, capturing multi-modal action distributions in 6G network management.

Findings

Diffusion-QL outperforms state-of-the-art DRL baselines in simulations.

The method effectively models complex, multi-modal action distributions.

Demonstrates robustness in managing diverse QoS requirements.

Abstract

Dynamic resource allocation in O-RAN is critical for managing the conflicting QoS requirements of 6G network slices. Conventional reinforcement learning agents often fail in this domain, as their unimodal policy structures cannot model the multi-modal nature of optimal allocation strategies. This paper introduces Diffusion Q-Learning (Diffusion-QL), a novel framework that represents the policy as a conditional diffusion model. Our approach generates resource allocation actions by iteratively reversing a noising process, with each step guided by the gradient of a learned Q-function. This method enables the policy to learn and sample from the complex distribution of near-optimal actions. Simulations demonstrate that the Diffusion-QL approach consistently outperforms state-of-the-art DRL baselines, offering a robust solution for the intricate resource management challenges in next-generation wireless networks.