Three-Dimensional Trajectory Design for Multi-User MISO UAV Communications: A Deep Reinforcement Learning Approach

TL;DR

This paper introduces a deep reinforcement learning method for optimizing the 3D trajectory of UAVs in multi-user MISO communication systems to minimize data transmission time in urban environments.

Contribution

It develops a novel DRL-based trajectory design approach using a deep deterministic policy gradient algorithm with pheromone-based state representation for urban UAV communications.

Findings

DRL-TDCTM outperforms baseline methods in simulations.

The approach effectively adapts UAV trajectories in complex urban environments.

Simulation results demonstrate reduced transmission completion time.

Abstract

In this paper, we investigate a multi-user downlink multiple-input single-output (MISO) unmanned aerial vehicle (UAV) communication system, where a multi-antenna UAV is employed to serve multiple ground terminals. Unlike existing approaches focus only on a simplified two-dimensional scenario, this paper considers a three-dimensional (3D) urban environment, where the UAV's 3D trajectory is designed to minimize data transmission completion time subject to practical throughput and flight movement constraints. Specifically, we propose a deep reinforcement learning (DRL)-based trajectory design for completion time minimization (DRL-TDCTM), which is developed from a deep deterministic policy gradient algorithm. In particular, to represent the state information of UAV and environment, we set an additional information, i.e., the merged pheromone, as a reference of reward which facilitates the algorithm design. By interacting with the external environment in the corresponding Markov decision process, the proposed algorithm can continuously and adaptively learn how to adjust the UAV's movement strategy. Finally, simulation results show the superiority of the proposed DRL-TDCTM algorithm over the conventional baseline methods.