Joint Path planning and Power Allocation of a Cellular-Connected UAV using Apprenticeship Learning via Deep Inverse Reinforcement Learning

TL;DR

This paper presents a deep inverse reinforcement learning approach for joint path planning and power allocation of cellular-connected UAVs, achieving expert-level performance and robustness in unseen scenarios.

Contribution

It introduces an apprenticeship learning method using IRL with deep RL for UAV path and power optimization, outperforming behavioral cloning.

Findings

Achieves expert-level performance in UAV path planning and power control.

Maintains robustness in unseen scenarios unlike behavioral cloning.

Demonstrates effectiveness through simulations and numerical analysis.

Abstract

This paper investigates an interference-aware joint path planning and power allocation mechanism for a cellular-connected unmanned aerial vehicle (UAV) in a sparse suburban environment. The UAV's goal is to fly from an initial point and reach a destination point by moving along the cells to guarantee the required quality of service (QoS). In particular, the UAV aims to maximize its uplink throughput and minimize the level of interference to the ground user equipment (UEs) connected to the neighbor cellular BSs, considering the shortest path and flight resource limitation. Expert knowledge is used to experience the scenario and define the desired behavior for the sake of the agent (i.e., UAV) training. To solve the problem, an apprenticeship learning method is utilized via inverse reinforcement learning (IRL) based on both Q-learning and deep reinforcement learning (DRL). The performance of this method is compared to learning from a demonstration technique called behavioral cloning (BC) using a supervised learning approach. Simulation and numerical results show that the proposed approach can achieve expert-level performance. We also demonstrate that, unlike the BC technique, the performance of our proposed approach does not degrade in unseen situations.