AdverSAR: Adversarial Search and Rescue via Multi-Agent Reinforcement Learning

TL;DR

This paper introduces AdverSAR, a multi-agent reinforcement learning approach enabling autonomous robots to coordinate effectively in adversarial environments for search and rescue missions, especially when communication is limited or compromised.

Contribution

It proposes a novel adversarial MARL framework using hierarchical meta-learning for decentralized coordination in hostile environments, improving autonomous SAR operations.

Findings

Effective coordination in adversarial settings demonstrated in grid-world simulations.

Hierarchical meta-learning enables emergent cooperative behaviors.

Robustness against communication disruptions shown in prototype environments.

Abstract

Search and Rescue (SAR) missions in remote environments often employ autonomous multi-robot systems that learn, plan, and execute a combination of local single-robot control actions, group primitives, and global mission-oriented coordination and collaboration. Often, SAR coordination strategies are manually designed by human experts who can remotely control the multi-robot system and enable semi-autonomous operations. However, in remote environments where connectivity is limited and human intervention is often not possible, decentralized collaboration strategies are needed for fully-autonomous operations. Nevertheless, decentralized coordination may be ineffective in adversarial environments due to sensor noise, actuation faults, or manipulation of inter-agent communication data. In this paper, we propose an algorithmic approach based on adversarial multi-agent reinforcement learning (MARL) that allows robots to efficiently coordinate their strategies in the presence of adversarial inter-agent communications. In our setup, the objective of the multi-robot team is to discover targets strategically in an obstacle-strewn geographical area by minimizing the average time needed to find the targets. It is assumed that the robots have no prior knowledge of the target locations, and they can interact with only a subset of neighboring robots at any time. Based on the centralized training with decentralized execution (CTDE) paradigm in MARL, we utilize a hierarchical meta-learning framework to learn dynamic team-coordination modalities and discover emergent team behavior under complex cooperative-competitive scenarios. The effectiveness of our approach is demonstrated on a collection of prototype grid-world environments with different specifications of benign and adversarial agents, target locations, and agent rewards.