Cooperative and Asynchronous Transformer-based Mission Planning for Heterogeneous Teams of Mobile Robots

TL;DR

This paper introduces CATMiP, a novel transformer-based framework for cooperative mission planning in heterogeneous mobile robot teams, effectively handling communication constraints and enabling scalable, robust multi-agent decision-making.

Contribution

The paper presents a new asynchronous transformer architecture and a macro-action model for multi-agent reinforcement learning in heterogeneous robot teams, improving scalability and robustness.

Findings

CATMiP outperforms planning-based methods in efficiency and scalability.

The framework is robust to communication dropouts and input noise.

It generalizes well to larger environments and varying team sizes.

Abstract

Cooperative mission planning for heterogeneous teams of mobile robots presents a unique set of challenges, particularly when operating under communication constraints and limited computational resources. To address these challenges, we propose the Cooperative and Asynchronous Transformer-based Mission Planning (CATMiP) framework, which leverages multi-agent reinforcement learning (MARL) to coordinate distributed decision making among agents with diverse sensing, motion, and actuation capabilities, operating under sporadic ad hoc communication. A Class-based Macro-Action Decentralized Partially Observable Markov Decision Process (CMacDec-POMDP) is also formulated to effectively model asynchronous decision-making for heterogeneous teams of agents. The framework utilizes an asynchronous centralized training and distributed execution scheme, enabled by the proposed Asynchronous Multi-Agent Transformer (AMAT) architecture. This design allows a single trained model to generalize to larger environments and accommodate varying team sizes and compositions. We evaluate CATMiP in a 2D grid-world simulation environment and compare its performance against planning-based exploration methods. Results demonstrate CATMiP's superior efficiency, scalability, and robustness to communication dropouts and input noise, highlighting its potential for real-world heterogeneous mobile robot systems. The code is available at https://github.com/mylad13/CATMiP