Integrated Sensing and Communications for Low-Altitude Economy: A Deep Reinforcement Learning Approach

TL;DR

This paper introduces Deep LAE-ISAC, a deep reinforcement learning-based scheme for integrated sensing and communications in low-altitude airspace, optimizing UAV trajectories and beamforming to maximize communication while ensuring sensing and safety constraints.

Contribution

It develops a novel DRL-based framework with specialized reward, action policy, and experience replay mechanisms for the LAE integrated sensing and communication system.

Findings

DeepLSC outperforms benchmarks in sum-rate and constraint satisfaction.

Faster convergence and robustness demonstrated through simulations.

Hierarchical experience replay and symmetric augmentation improve learning efficiency.

Abstract

This paper studies an integrated sensing and communications (ISAC) system for low-altitude economy (LAE), where a ground base station (GBS) provides communication and navigation services for authorized unmanned aerial vehicles (UAVs), while sensing the low-altitude airspace to monitor the unauthorized mobile target. The expected communication sum-rate over a given flight period is maximized by jointly optimizing the beamforming at the GBS and UAVs' trajectories, subject to the constraints on the average signal-to-noise ratio requirement for sensing, the flight mission and collision avoidance of UAVs, as well as the maximum transmit power at the GBS. Typically, this is a sequential decision-making problem with the given flight mission. Thus, we transform it to a specific Markov decision process (MDP) model called episode task. Based on this modeling, we propose a novel LAE-oriented ISAC scheme, referred to as Deep LAE-ISAC (DeepLSC), by leveraging the deep reinforcement learning (DRL) technique. In DeepLSC, a reward function and a new action selection policy termed constrained noise-exploration policy are judiciously designed to fulfill various constraints. To enable efficient learning in episode tasks, we develop a hierarchical experience replay mechanism, where the gist is to employ all experiences generated within each episode to jointly train the neural network. Besides, to enhance the convergence speed of DeepLSC, a symmetric experience augmentation mechanism, which simultaneously permutes the indexes of all variables to enrich available experience sets, is proposed. Simulation results demonstrate that compared with benchmarks, DeepLSC yields a higher sum-rate while meeting the preset constraints, achieves faster convergence, and is more robust against different settings.