Learning Long-Range Perception Using Self-Supervision from Short-Range Sensors and Odometry

TL;DR

This paper presents a self-supervised method enabling robots to predict short-range sensor outputs from long-range sensor data, improving obstacle detection and avoidance without manual labeling.

Contribution

It introduces a general self-supervised framework for long-range perception using short-range sensors and odometry, validated on real and simulated robotic scenarios.

Findings

Accurately predicts obstacle presence from camera data

Demonstrates robustness across different conditions

Enables obstacle avoidance using only predicted outputs

Abstract

We introduce a general self-supervised approach to predict the future outputs of a short-range sensor (such as a proximity sensor) given the current outputs of a long-range sensor (such as a camera); we assume that the former is directly related to some piece of information to be perceived (such as the presence of an obstacle in a given position), whereas the latter is information-rich but hard to interpret directly. We instantiate and implement the approach on a small mobile robot to detect obstacles at various distances using the video stream of the robot's forward-pointing camera, by training a convolutional neural network on automatically-acquired datasets. We quantitatively evaluate the quality of the predictions on unseen scenarios, qualitatively evaluate robustness to different operating conditions, and demonstrate usage as the sole input of an obstacle-avoidance controller. We additionally instantiate the approach on a different simulated scenario with complementary characteristics, to exemplify the generality of our contribution.