Robot Navigation in a Crowd by Integrating Deep Reinforcement Learning and Online Planning

TL;DR

This paper introduces SG-DQN, a graph-based deep reinforcement learning approach that combines social attention and online planning to enable mobile robots to navigate crowds efficiently and safely, with high success rates and reduced computational costs.

Contribution

The paper presents a novel graph-based deep reinforcement learning method integrating social attention and online planning for improved crowd navigation.

Findings

Success rate exceeds 0.99 in crowd navigation tasks

Achieves comparable or better performance than state-of-the-art methods

Requires less than half the computational cost of previous approaches

Abstract

It is still an open and challenging problem for mobile robots navigating along time-efficient and collision-free paths in a crowd. The main challenge comes from the complex and sophisticated interaction mechanism, which requires the robot to understand the crowd and perform proactive and foresighted behaviors. Deep reinforcement learning is a promising solution to this problem. However, most previous learning methods incur a tremendous computational burden. To address these problems, we propose a graph-based deep reinforcement learning method, SG-DQN, that (i) introduces a social attention mechanism to extract an efficient graph representation for the crowd-robot state; (ii) directly evaluates the coarse q-values of the raw state with a learned dueling deep Q network(DQN); and then (iii) refines the coarse q-values via online planning on possible future trajectories. The experimental results indicate that our model can help the robot better understand the crowd and achieve a high success rate of more than 0.99 in the crowd navigation task. Compared against previous state-of-the-art algorithms, our algorithm achieves an equivalent, if not better, performance while requiring less than half of the computational cost.