Learning Observation-Based Certifiable Safe Policy for Decentralized Multi-Robot Navigation

TL;DR

This paper introduces a novel safe multi-robot navigation framework combining reinforcement learning and control barrier functions, ensuring safety with high probability using sensor data and a joint training approach, validated in simulation and real-world tests.

Contribution

It presents a joint training framework integrating RL policies with CBF-based safety optimization, improving safety and success rates in multi-robot navigation.

Findings

Higher success rate in multi-robot tasks

Effective safety guarantees with sensor-only data

Validated in both simulation and real-world scenarios

Abstract

Safety is of great importance in multi-robot navigation problems. In this paper, we propose a control barrier function (CBF) based optimizer that ensures robot safety with both high probability and flexibility, using only sensor measurement. The optimizer takes action commands from the policy network as initial values and then provides refinement to drive the potentially dangerous ones back into safe regions. With the help of a deep transition model that predicts the evolution of surrounding dynamics and the consequences of different actions, the CBF module can guide the optimization in a reasonable time horizon. We also present a novel joint training framework that improves the cooperation between the Reinforcement Learning (RL) based policy and the CBF-based optimizer both in training and inference procedures by utilizing reward feedback from the CBF module. We observe that the policy using our method can achieve a higher success rate while maintaining the safety of multiple robots in significantly fewer episodes compared with other methods. Experiments are conducted in multiple scenarios both in simulation and the real world, the results demonstrate the effectiveness of our method in maintaining the safety of multi-robot navigation. Code is available at \url{https://github.com/YuxiangCui/MARL-OCBF