Human-Aware Robot Navigation via Reinforcement Learning with Hindsight Experience Replay and Curriculum Learning

TL;DR

This paper introduces a reinforcement learning approach enhanced with hindsight experience replay and curriculum learning to enable robots to navigate safely and efficiently in crowded environments without needing expert demonstration data.

Contribution

It presents a novel RL-based method combining HER and CL for human-aware navigation, eliminating the dependence on costly demonstration data.

Findings

Successfully learned collision-free navigation in dense crowds

Achieved efficient training without demonstration data

Validated effectiveness in simulated environments

Abstract

In recent years, the growing demand for more intelligent service robots is pushing the development of mobile robot navigation algorithms to allow safe and efficient operation in a dense crowd. Reinforcement learning (RL) approaches have shown superior ability in solving sequential decision making problems, and recent work has explored its potential to learn navigation polices in a socially compliant manner. However, the expert demonstration data used in existing methods is usually expensive and difficult to obtain. In this work, we consider the task of training an RL agent without employing the demonstration data, to achieve efficient and collision-free navigation in a crowded environment. To address the sparse reward navigation problem, we propose to incorporate the hindsight experience replay (HER) and curriculum learning (CL) techniques with RL to efficiently learn the optimal navigation policy in the dense crowd. The effectiveness of our method is validated in a simulated crowd-robot coexisting environment. The results demonstrate that our method can effectively learn human-aware navigation without requiring additional demonstration data.