Optical-Flow based Self-Supervised Learning of Obstacle Appearance applied to MAV Landing

TL;DR

This paper presents a self-supervised learning approach using optical flow cues to enable MAVs to detect obstacles from still images, reducing the need for movement during navigation and improving safety in narrow spaces.

Contribution

It introduces a novel SSL setup that learns obstacle appearance from optical flow, allowing obstacle detection from single images without movement.

Findings

Successfully demonstrated in offline tests with images.

Validated in flight with real MAVs.

Enabled pixel-wise obstacle segmentation.

Abstract

Monocular optical flow has been widely used to detect obstacles in Micro Air Vehicles (MAVs) during visual navigation. However, this approach requires significant movement, which reduces the efficiency of navigation and may even introduce risks in narrow spaces. In this paper, we introduce a novel setup of self-supervised learning (SSL), in which optical flow cues serve as a scaffold to learn the visual appearance of obstacles in the environment. We apply it to a landing task, in which initially 'surface roughness' is estimated from the optical flow field in order to detect obstacles. Subsequently, a linear regression function is learned that maps appearance features represented by texton distributions to the roughness estimate. After learning, the MAV can detect obstacles by just analyzing a still image. This allows the MAV to search for a landing spot without moving. We first demonstrate this principle to work with offline tests involving images captured from an on-board camera, and then demonstrate the principle in flight. Although surface roughness is a property of the entire flow field in the global image, the appearance learning even allows for the pixel-wise segmentation of obstacles.