Generative Learning Enhanced Intelligent Resource Management for Cell-Free Delay Deterministic Communications

TL;DR

This paper introduces a novel offline pretraining framework using virtual CMDP modeling and evidence-aware Gaussian Mixture Models to enhance resource management in CF-MIMO systems, improving energy efficiency and delay compliance.

Contribution

It proposes a new pretraining approach for safe deep reinforcement learning in CF-MIMO, addressing data sparsity and distribution drift, with significant efficiency and performance gains.

Findings

Pretrained agent achieves twice the initial energy efficiency.

Maintains a 1% delay violation rate.

Converges to 4.7% higher energy efficiency with fewer exploration steps.

Abstract

Cell-free multiple-input multiple-output (CF-MIMO) architecture significantly enhances wireless network performance, offering a promising solution for delay-sensitive applications. This paper investigates the resource allocation problem in CF-MIMO systems, aiming to maximize energy efficiency (EE) while satisfying delay violation rate constraint. We design a Proximal Policy Optimization (PPO) with a primal-dual method to solve it. To address the low sample efficiency and safety risks caused by cold-start of the designed safe deep reinforcement learning (DRL) method, we propose a novel offline pretraining framework based on virtual constrained Markov decision process (CMDP) modeling. The virtual CMDP consists of reward and cost prediction module, initial-state distribution module and state transition module. Notably, we propose an evidence-aware conditional Gaussian Mixture Model (EA-CGMM) inference approach to mitigate data sparsity and distribution drift issues in state transition modeling. Simulation results demonstrate the effectiveness of CMDP modeling and validate the safety and efficiency of the proposed pretraining framework. Specifically, compared with non-pretrained baseline, the agent pretrained through our proposed framework achieves twice the initial EE and maintains a low delay constraint violation rate of $1\%$, while ultimately converging to an EE that is $4.7\%$ higher with a $50\%$ reduction in exploration steps. Additionally, our proposed pretraining framework implementation exhibits comparable performance to the SOTA diffusion model-based implementation, while achieving a $14$-fold reduction in computational complexity.