Dynamic one-time delivery of critical data by small and sparse UAV swarms: a model problem for MARL scaling studies

TL;DR

This paper explores the use of Multi-Agent Reinforcement Learning for decentralized UAV control in critical data relay tasks, highlighting scalability challenges and providing a benchmark environment.

Contribution

Introduces a family of deterministic games for MARL scalability studies and proposes a baseline policy for UAV data relay tasks.

Findings

Off-the-shelf MARL algorithms perform well with few agents

Scalability issues emerge as agent count increases

Source code and animations are publicly available

Abstract

This work studies the application of Multi-Agent Reinforcement Learning (MARL) to decentralized control of unmanned aerial vehicles to relay a critical data package to a known position. For this purpose, a family of deterministic games is introduced, designed for MARL scaling studies. A robust baseline policy is proposed which restricts agent motion and applies Dijkstra's shortest path algorithm. Computational experiment results show that two off-the-shelf MARL algorithms perform competitively with the baseline for a small number of agents, but face scalability issues as the number of agents increases. Source code and animations are available online at https://github.com/mikapersson/Information-Relaying.