Meta-Reinforcement Learning for Trajectory Design in Wireless UAV Networks

TL;DR

This paper introduces a meta-reinforcement learning approach to optimize drone trajectories in dynamic wireless networks, enabling quick adaptation and improved performance in unseen environments.

Contribution

It proposes a meta-learning algorithm that enhances reinforcement learning for drone trajectory design, achieving faster convergence and better adaptability in unpredictable network conditions.

Findings

25% faster convergence in training

10% improvement in communication performance

27% increase in serving over 50% user requests

Abstract

In this paper, the design of an optimal trajectory for an energy-constrained drone operating in dynamic network environments is studied. In the considered model, a drone base station (DBS) is dispatched to provide uplink connectivity to ground users whose demand is dynamic and unpredictable. In this case, the DBS's trajectory must be adaptively adjusted to satisfy the dynamic user access requests. To this end, a meta-learning algorithm is proposed in order to adapt the DBS's trajectory when it encounters novel environments, by tuning a reinforcement learning (RL) solution. The meta-learning algorithm provides a solution that adapts the DBS in novel environments quickly based on limited former experiences. The meta-tuned RL is shown to yield a faster convergence to the optimal coverage in unseen environments with a considerably low computation complexity, compared to the baseline policy gradient algorithm. Simulation results show that, the proposed meta-learning solution yields a 25% improvement in the convergence speed, and about 10% improvement in the DBS' communication performance, compared to a baseline policy gradient algorithm. Meanwhile, the probability that the DBS serves over 50% of user requests increases about 27%, compared to the baseline policy gradient algorithm.