Safe Multi-Agent Reinforcement Learning for Behavior-Based Cooperative Navigation

TL;DR

This paper introduces a safe multi-agent reinforcement learning framework for cooperative navigation of mobile robots that uses a single target for the formation's centroid, ensuring collision-free operation and faster convergence.

Contribution

It presents the first MARL approach for cooperative navigation without individual targets, integrating MPC safety filters for improved safety and training efficiency.

Findings

Zero collisions achieved in simulation and real-world tests

Faster convergence during training with MPC safety filters

Effective deployment on real robots during early training stages

Abstract

In this paper, we address the problem of behavior-based cooperative navigation of mobile robots using safe multi-agent reinforcement learning~(MARL). Our work is the first to focus on cooperative navigation without individual reference targets for the robots, using a single target for the formation's centroid. This eliminates the complexities involved in having several path planners to control a team of robots. To ensure safety, our MARL framework uses model predictive control (MPC) to prevent actions that could lead to collisions during training and execution. We demonstrate the effectiveness of our method in simulation and on real robots, achieving safe behavior-based cooperative navigation without using individual reference targets, with zero collisions, and faster target reaching compared to baselines. Finally, we study the impact of MPC safety filters on the learning process, revealing that we achieve faster convergence during training and we show that our approach can be safely deployed on real robots, even during early stages of the training.