SREC: Proactive Self-Remedy of Energy-Constrained UAV-Based Networks via Deep Reinforcement Learning

TL;DR

This paper introduces SREC-DRL, a deep reinforcement learning approach for proactively managing energy-constrained UAV networks by relocating UAVs before they deplete, thereby improving user satisfaction.

Contribution

It proposes a novel proactive control policy using deep reinforcement learning to optimize UAV relocation before energy depletion, enhancing network performance.

Findings

SREC-DRL achieves a 12.12% higher user satisfaction score compared to passive methods.

The approach effectively handles continuous state and action spaces with DDPG.

Proactive UAV management improves network resilience and user experience.

Abstract

Energy-aware control for multiple unmanned aerial vehicles (UAVs) is one of the major research interests in UAV based networking. Yet few existing works have focused on how the network should react around the timing when the UAV lineup is changed. In this work, we study proactive self-remedy of energy-constrained UAV networks when one or more UAVs are short of energy and about to quit for charging. We target at an energy-aware optimal UAV control policy which proactively relocates the UAVs when any UAV is about to quit the network, rather than passively dispatches the remaining UAVs after the quit. Specifically, a deep reinforcement learning (DRL)-based self remedy approach, named SREC-DRL, is proposed to maximize the accumulated user satisfaction scores for a certain period within which at least one UAV will quit the network. To handle the continuous state and action space in the problem, the state-of-the-art algorithm of the actor-critic DRL, i.e., deep deterministic policy gradient (DDPG), is applied with better convergence stability. Numerical results demonstrate that compared with the passive reaction method, the proposed SREC-DRL approach shows a $12.12\%$ gain in accumulative user satisfaction score during the remedy period.