Persistent self-supervised learning principle: from stereo to monocular vision for obstacle avoidance

TL;DR

This paper introduces a persistent self-supervised learning strategy enabling a flying robot to switch from stereo to monocular vision for obstacle avoidance, ensuring continuous operation even if the original sensor cue fails.

Contribution

It presents a novel persistent SSL approach that allows robots to adaptively switch sensor cues, demonstrated on a drone using stereo vision to train monocular distance estimation.

Findings

Successful obstacle avoidance with monocular vision after training

Effective handling of feedback-induced data bias

Feasibility demonstrated in real-world drone experiments

Abstract

Self-Supervised Learning (SSL) is a reliable learning mechanism in which a robot uses an original, trusted sensor cue for training to recognize an additional, complementary sensor cue. We study for the first time in SSL how a robot's learning behavior should be organized, so that the robot can keep performing its task in the case that the original cue becomes unavailable. We study this persistent form of SSL in the context of a flying robot that has to avoid obstacles based on distance estimates from the visual cue of stereo vision. Over time it will learn to also estimate distances based on monocular appearance cues. A strategy is introduced that has the robot switch from stereo vision based flight to monocular flight, with stereo vision purely used as 'training wheels' to avoid imminent collisions. This strategy is shown to be an effective approach to the 'feedback-induced data bias' problem as also experienced in learning from demonstration. Both simulations and real-world experiments with a stereo vision equipped AR drone 2.0 show the feasibility of this approach, with the robot successfully using monocular vision to avoid obstacles in a 5 x 5 room. The experiments show the potential of persistent SSL as a robust learning approach to enhance the capabilities of robots. Moreover, the abundant training data coming from the own sensors allows to gather large data sets necessary for deep learning approaches.