HeR-DRL:Heterogeneous Relational Deep Reinforcement Learning for Decentralized Multi-Robot Crowd Navigation

TL;DR

This paper introduces HeR-DRL, a heterogeneous relational deep reinforcement learning approach utilizing a customized GNN to enhance decentralized multi-robot crowd navigation, significantly improving safety and comfort over existing methods.

Contribution

The paper presents a novel heterogeneous GNN-based framework for modeling diverse interactions in multi-robot crowd navigation, improving policy learning and navigation performance.

Findings

HeR-DRL outperforms state-of-the-art algorithms in safety and comfort metrics.

The heterogeneous relation graph effectively models diverse robot-crowd interactions.

Experimental results validate the importance of interaction heterogeneity in crowd navigation.

Abstract

Crowd navigation has received significant research attention in recent years, especially DRL-based methods. While single-robot crowd scenarios have dominated research, they offer limited applicability to real-world complexities. The heterogeneity of interaction among multiple agent categories, like in decentralized multi-robot pedestrian scenarios, are frequently disregarded. This "interaction blind spot" hinders generalizability and restricts progress towards robust navigation algorithms. In this paper, we propose a heterogeneous relational deep reinforcement learning(HeR-DRL), based on customised heterogeneous GNN, in order to improve navigation strategies in decentralized multi-robot crowd navigation. Firstly, we devised a method for constructing robot-crowd heterogenous relation graph that effectively simulates the heterogeneous pair-wise interaction relationships. We proposed a new heterogeneous graph neural network for transferring and aggregating the heterogeneous state information. Finally, we incorporate the encoded information into deep reinforcement learning to explore the optimal policy. HeR-DRL are rigorously evaluated through comparing it to state-of-the-art algorithms in both single-robot and multi-robot circle crowssing scenario. The experimental results demonstrate that HeR-DRL surpasses the state-of-the-art approaches in overall performance, particularly excelling in safety and comfort metrics. This underscores the significance of interaction heterogeneity for crowd navigation. The source code will be publicly released in https://github.com/Zhouxy-Debugging-Den/HeR-DRL.