Communication-Aware Multi-Agent Reinforcement Learning for Decentralized Cooperative UAV Deployment

TL;DR

This paper introduces a graph-based multi-agent reinforcement learning framework for decentralized UAV swarm deployment, enabling efficient communication-aware cooperation under partial observability and limited links.

Contribution

It presents a novel architecture with attention modules for encoding local states and messages, trained via centralized critic and decentralized execution, applicable to cooperative and adversarial UAV tasks.

Findings

Achieves 74% coverage in DroneConnect with limited communication

Generalizes to unseen team sizes without fine-tuning

Improves win rate in adversarial UAV engagement

Abstract

Autonomous Unmanned Aerial Vehicle (UAV) swarms are increasingly used as rapidly deployable aerial relays and sensing platforms, yet practical deployments must operate under partial observability and intermittent peer-to-peer links. We present a graph-based multi-agent reinforcement learning framework trained under centralized training with decentralized execution (CTDE): a centralized critic and global state are available only during training, while each UAV executes a shared policy using local observations and messages from nearby neighbors. Our architecture encodes local agent state and nearby entities with an agent-entity attention module, and aggregates inter-UAV messages with neighbor self-attention over a distance-limited communication graph. We evaluate primarily on a cooperative relay deployment task (DroneConnect) and secondarily on an adversarial engagement task (DroneCombat). In DroneConnect, the proposed method achieves high coverage under restricted communication and partial observation (e.g. 74% coverage with M = 5 UAVs and N = 10 nodes) while remaining competitive with a mixed-integer linear programming (MILP) optimization-based offline upper bound, and it generalizes to unseen team sizes without fine-tuning. In the adversarial setting, the same framework transfers without architectural changes and improves win rate over non-communicating baselines.