Intrinsic-Motivation Multi-Robot Social Formation Navigation with Coordinated Exploration

TL;DR

This paper introduces a novel intrinsic-motivation multi-robot RL algorithm for social formation navigation, effectively handling unpredictable pedestrian behaviors and improving coordinated exploration and policy learning.

Contribution

It proposes a self-learning intrinsic reward mechanism with dual-sampling and two-time-scale updates, advancing multi-robot RL for social navigation.

Findings

Outperforms existing methods on social formation benchmarks

Enhances coordinated exploration efficiency

Demonstrates robustness in dynamic pedestrian environments

Abstract

This paper investigates the application of reinforcement learning (RL) to multi-robot social formation navigation, a critical capability for enabling seamless human-robot coexistence. While RL offers a promising paradigm, the inherent unpredictability and often uncooperative dynamics of pedestrian behavior pose substantial challenges, particularly concerning the efficiency of coordinated exploration among robots. To address this, we propose a novel coordinated-exploration multi-robot RL algorithm introducing an intrinsic motivation exploration. Its core component is a self-learning intrinsic reward mechanism designed to collectively alleviate policy conservatism. Moreover, this algorithm incorporates a dual-sampling mode within the centralized training and decentralized execution framework to enhance the representation of both the navigation policy and the intrinsic reward, leveraging a two-time-scale update rule to decouple parameter updates. Empirical results on social formation navigation benchmarks demonstrate the proposed algorithm's superior performance over existing state-of-the-art methods across crucial metrics. Our code and video demos are available at: https://github.com/czxhunzi/CEMRRL.