Uncertainty-Aware Reinforcement Learning for Collision Avoidance

TL;DR

This paper introduces an uncertainty-aware reinforcement learning algorithm for collision avoidance in robots, enabling cautious exploration and safe learning in unknown environments by estimating collision probabilities and uncertainties.

Contribution

It presents a novel model-based learning method that incorporates uncertainty estimation to improve safety during robot training in collision avoidance tasks.

Findings

Effectively minimizes dangerous collisions during training

Enables cautious exploration in unfamiliar environments

Demonstrates success on simulated and real-world robots

Abstract

Reinforcement learning can enable complex, adaptive behavior to be learned automatically for autonomous robotic platforms. However, practical deployment of reinforcement learning methods must contend with the fact that the training process itself can be unsafe for the robot. In this paper, we consider the specific case of a mobile robot learning to navigate an a priori unknown environment while avoiding collisions. In order to learn collision avoidance, the robot must experience collisions at training time. However, high-speed collisions, even at training time, could damage the robot. A successful learning method must therefore proceed cautiously, experiencing only low-speed collisions until it gains confidence. To this end, we present an uncertainty-aware model-based learning algorithm that estimates the probability of collision together with a statistical estimate of uncertainty. By formulating an uncertainty-dependent cost function, we show that the algorithm naturally chooses to proceed cautiously in unfamiliar environments, and increases the velocity of the robot in settings where it has high confidence. Our predictive model is based on bootstrapped neural networks using dropout, allowing it to process raw sensory inputs from high-bandwidth sensors such as cameras. Our experimental evaluation demonstrates that our method effectively minimizes dangerous collisions at training time in an obstacle avoidance task for a simulated and real-world quadrotor, and a real-world RC car. Videos of the experiments can be found at https://sites.google.com/site/probcoll.