A Lyapunov-Guided Diffusion-Based Reinforcement Learning Approach for UAV-Assisted Vehicular Networks with Delayed CSI Feedback

TL;DR

This paper introduces a Lyapunov-guided diffusion-based reinforcement learning method for UAV-assisted vehicular networks, optimizing resource allocation and UAV trajectories under delayed CSI feedback to enhance communication efficiency and energy use.

Contribution

It proposes a novel D3PG algorithm integrating diffusion models for joint channel, power, and altitude optimization considering CSI delay, advancing UAV-assisted vehicular network management.

Findings

D3PG outperforms existing benchmarks in simulation tests.

The approach effectively manages energy constraints and CSI delays.

Simulation results show significant improvements in communication rates.

Abstract

Low altitude uncrewed aerial vehicles (UAVs) are expected to facilitate the development of aerial-ground integrated intelligent transportation systems and unlocking the potential of the emerging low-altitude economy. However, several critical challenges persist, including the dynamic optimization of network resources and UAV trajectories, limited UAV endurance, and imperfect channel state information (CSI). In this paper, we offer new insights into low-altitude economy networking by exploring intelligent UAV-assisted vehicle-to-everything communication strategies aligned with UAV energy efficiency. Particularly, we formulate an optimization problem of joint channel allocation, power control, and flight altitude adjustment in UAV-assisted vehicular networks. Taking CSI feedback delay into account, our objective is to maximize the vehicle-to-UAV communication sum rate while satisfying the UAV's long-term energy constraint. To this end, we first leverage Lyapunov optimization to decompose the original long-term problem into a series of per-slot deterministic subproblems. We then propose a diffusion-based deep deterministic policy gradient (D3PG) algorithm, which innovatively integrates diffusion models to determine optimal channel allocation, power control, and flight altitude adjustment decisions. Through extensive simulations using real-world vehicle mobility traces, we demonstrate the superior performance of the proposed D3PG algorithm compared to existing benchmark solutions.