Learning Implicit Social Navigation Behavior using Deep Inverse Reinforcement Learning

TL;DR

This paper introduces S-MEDIRL, a deep inverse reinforcement learning algorithm that enables robots to learn social navigation behaviors by predicting cost maps from trajectory data and scene geometry, outperforming rule-based methods.

Contribution

The paper presents a novel deep inverse reinforcement learning approach for social navigation that extrapolates beyond demonstrations to encode scene navigability and social behaviors.

Findings

Robot learns to yield and avoid deadlocks in simulation

S-MEDIRL outperforms ORCA and rule-based agents

Effective in narrow crossing scenarios

Abstract

This paper reports on learning a reward map for social navigation in dynamic environments where the robot can reason about its path at any time, given agents' trajectories and scene geometry. Humans navigating in dense and dynamic indoor environments often work with several implied social rules. A rule-based approach fails to model all possible interactions between humans, robots, and scenes. We propose a novel Smooth Maximum Entropy Deep Inverse Reinforcement Learning (S-MEDIRL) algorithm that can extrapolate beyond expert demos to better encode scene navigability from few-shot demonstrations. The agent learns to predict the cost maps reasoning on trajectory data and scene geometry. The agent samples a trajectory that is then executed using a local crowd navigation controller. We present results in a photo-realistic simulation environment, with a robot and a human navigating a narrow crossing scenario. The robot implicitly learns to exhibit social behaviors such as yielding to oncoming traffic and avoiding deadlocks. We compare the proposed approach to the popular model-based crowd navigation algorithm ORCA and a rule-based agent that exhibits yielding.