Disentangling Uncertainty for Safe Social Navigation using Deep Reinforcement Learning

TL;DR

This paper presents a novel deep reinforcement learning framework for safe social robot navigation that estimates different types of uncertainty to improve decision-making and collision avoidance in pedestrian environments.

Contribution

It introduces a method combining aleatoric, epistemic, and predictive uncertainty estimation into DRL for social navigation, using ODV and dropout with PPO.

Findings

Improved training performance with ODV and dropout.

MC-dropout better detects perturbations and correlates with uncertainty types.

Enhanced navigation safety with conservative behaviors in uncertain situations.

Abstract

Autonomous mobile robots are increasingly used in pedestrian-rich environments where safe navigation and appropriate human interaction are crucial. While Deep Reinforcement Learning (DRL) enables socially integrated robot behavior, challenges persist in novel or perturbed scenarios to indicate when and why the policy is uncertain. Unknown uncertainty in decision-making can lead to collisions or human discomfort and is one reason why safe and risk-aware navigation is still an open problem. This work introduces a novel approach that integrates aleatoric, epistemic, and predictive uncertainty estimation into a DRL navigation framework for policy distribution uncertainty estimates. We, therefore, incorporate Observation-Dependent Variance (ODV) and dropout into the Proximal Policy Optimization (PPO) algorithm. For different types of perturbations, we compare the ability of deep ensembles and Monte-Carlo dropout (MC-dropout) to estimate the uncertainties of the policy. In uncertain decision-making situations, we propose to change the robot's social behavior to conservative collision avoidance. The results show improved training performance with ODV and dropout in PPO and reveal that the training scenario has an impact on the generalization. In addition, MC-dropout is more sensitive to perturbations and correlates the uncertainty type to the perturbation better. With the safe action selection, the robot can navigate in perturbed environments with fewer collisions.