Learning with Dynamics: Autonomous Regulation of UAV Based Communication Networks with Dynamic UAV Crew

TL;DR

This paper explores reinforcement learning strategies for adaptively managing UAV-based communication networks with dynamically changing UAV sets, addressing both reactive and proactive approaches in different scenarios.

Contribution

It introduces RL-based adaptive regulation strategies for UCNs with dynamic UAV crews, including case studies and discussion of key challenges and solutions.

Findings

RL can effectively adapt to dynamic UAV sets in UCNs

Proactive strategies improve network stability in solar-powered UAVs

Case studies demonstrate the potential of various RL algorithms

Abstract

Unmanned Aerial Vehicle (UAV) based communication networks (UCNs) are a key component in future mobile networking. To handle the dynamic environments in UCNs, reinforcement learning (RL) has been a promising solution attributed to its strong capability of adaptive decision-making free of the environment models. However, most existing RL-based research focus on control strategy design assuming a fixed set of UAVs. Few works have investigated how UCNs should be adaptively regulated when the serving UAVs change dynamically. This article discusses RL-based strategy design for adaptive UCN regulation given a dynamic UAV set, addressing both reactive strategies in general UCNs and proactive strategies in solar-powered UCNs. An overview of the UCN and the RL framework is first provided. Potential research directions with key challenges and possible solutions are then elaborated. Some of our recent works are presented as case studies to inspire innovative ways to handle dynamic UAV crew with different RL algorithms.