Deep Reinforcement Learning based Autonomous Decision-Making for Cooperative UAVs: A Search and Rescue Real World Application

TL;DR

This paper introduces an integrated deep reinforcement learning framework for autonomous UAV search-and-rescue missions in indoor environments, demonstrating real-world deployment and competitive performance.

Contribution

It presents a novel end-to-end system combining DRL guidance, cooperative task allocation with GAT, and altitude stabilization techniques for indoor UAV SAR operations.

Findings

Successful real-world deployment on quad-rotors

Achieved first place in NATO-Sapience competition

Demonstrated reliable navigation in complex indoor environments

Abstract

This paper presents the first end-to-end framework that combines guidance, navigation, and centralised task allocation for multiple UAVs performing autonomous search-and-rescue (SAR) in GNSS-denied indoor environments. A Twin Delayed Deep Deterministic Policy Gradient controller is trained with an Artificial Potential Field (APF) reward that blends attractive and repulsive potentials with continuous control, accelerating convergence and yielding smoother, safer trajectories than distance-only baselines. Collaborative mission assignment is solved by a deep Graph Attention Network that, at each decision step, reasons over the drone-task graph to produce near-optimal allocations with negligible on-board compute. To arrest the notorious Z-drift of indoor LiDAR-SLAM, we fuse depth-camera altimetry with IMU vertical velocity in a lightweight complementary filter, giving centimetre-level altitude stability without external beacons. The resulting system was deployed on two 1m-class quad-rotors and flight-tested in a cluttered, multi-level disaster mock-up designed for the NATO-Sapience Autonomous Cooperative Drone Competition. Compared with prior DRL guidance that remains largely in simulation, our framework demonstrates an ability to navigate complex indoor environments, securing first place in the 2024 event. These results demonstrate that APF-shaped DRL and GAT-driven cooperation can translate to reliable real-world SAR operations.