RSS-Based UAV-BS 3-D Mobility Management via Policy Gradient Deep Reinforcement Learning

TL;DR

This paper proposes a model-free, RSS-based deep reinforcement learning approach for autonomous 3-D mobility management of UAV-mounted base stations, optimizing data rates without prior environmental knowledge.

Contribution

It introduces a policy gradient deep reinforcement learning method for UAV-BS mobility control that does not rely on environmental models or user data, enhancing flexibility and privacy.

Findings

UAV-BS adapts height based on environment type (high-rise vs. suburban).

The reward function influences the agent's speed and constraint adherence.

The approach achieves effective data rate optimization with minimal environmental information.

Abstract

We address the mobility management of an autonomous UAV-mounted base station (UAV-BS) that provides communication services to a cluster of users on the ground while the geographical characteristics (e.g., location and boundary) of the cluster, the geographical locations of the users, and the characteristics of the radio environment are unknown. UAVBS solely exploits the received signal strengths (RSS) from the users and accordingly chooses its (continuous) 3-D speed to constructively navigate, i.e., improving the transmitted data rate. To compensate for the lack of a model, we adopt policy gradient deep reinforcement learning. As our approach does not rely on any particular information about the users as well as the radio environment, it is flexible and respects the privacy concerns. Our experiments indicate that despite the minimum available information the UAV-BS is able to distinguish between high-rise (often non-line-of-sight dominant) and sub-urban (mainly line-of-sight dominant) environments such that in the former (resp. latter) it tends to reduce (resp. increase) its height and stays close (resp. far) to the cluster. We further observe that the choice of the reward function affects the speed and the ability of the agent to adhere to the problem constraints without affecting the delivered data rate.