Robot Perception enables Complex Navigation Behavior via Self-Supervised Learning

TL;DR

This paper introduces a self-supervised reinforcement learning approach that unifies perception systems for complex, goal-oriented robot navigation, demonstrating significant improvements in real-world driving scenarios under challenging conditions.

Contribution

It presents a novel method combining visual and motion perception data via self-supervision for active navigation, advancing beyond existing passive perception systems.

Findings

Achieves up to 80% success rate in extreme environmental conditions

Generalizes well to day-night cycles in real-world datasets

Outperforms vision-only systems significantly

Abstract

Learning visuomotor control policies in robotic systems is a fundamental problem when aiming for long-term behavioral autonomy. Recent supervised-learning-based vision and motion perception systems, however, are often separately built with limited capabilities, while being restricted to few behavioral skills such as passive visual odometry (VO) or mobile robot visual localization. Here we propose an approach to unify those successful robot perception systems for active target-driven navigation tasks via reinforcement learning (RL). Our method temporally incorporates compact motion and visual perception data - directly obtained using self-supervision from a single image sequence - to enable complex goal-oriented navigation skills. We demonstrate our approach on two real-world driving dataset, KITTI and Oxford RobotCar, using the new interactive CityLearn framework. The results show that our method can accurately generalize to extreme environmental changes such as day to night cycles with up to an 80% success rate, compared to 30% for a vision-only navigation systems.