Multi-robot Social-aware Cooperative Planning in Pedestrian Environments Using Multi-agent Reinforcement Learning

TL;DR

This paper introduces a novel multi-robot cooperative planning approach using multi-agent reinforcement learning, incorporating social-aware encoding and lookahead rewards to improve safety and efficiency in pedestrian environments.

Contribution

It proposes a social-aware multi-robot planner with a TSG-based encoder, K-step lookahead rewards, and an improved critic network, advancing multi-robot RL in pedestrian scenarios.

Findings

Effective in pedestrian environments with improved safety.

Reduces aggressive and unnatural robot motions.

Demonstrates superior performance in multi-group experiments.

Abstract

Safe and efficient co-planning of multiple robots in pedestrian participation environments is promising for applications. In this work, a novel multi-robot social-aware efficient cooperative planner that on the basis of off-policy multi-agent reinforcement learning (MARL) under partial dimension-varying observation and imperfect perception conditions is proposed. We adopt temporal-spatial graph (TSG)-based social encoder to better extract the importance of social relation between each robot and the pedestrians in its field of view (FOV). Also, we introduce K-step lookahead reward setting in multi-robot RL framework to avoid aggressive, intrusive, short-sighted, and unnatural motion decisions generated by robots. Moreover, we improve the traditional centralized critic network with multi-head global attention module to better aggregates local observation information among different robots to guide the process of individual policy update. Finally, multi-group experimental results verify the effectiveness of the proposed cooperative motion planner.