Self-Supervised Depth Completion for Active Stereo

TL;DR

This paper introduces a novel self-supervised depth completion method for active stereo systems, leveraging visual inertial SLAM and a new reconstruction loss to produce dense, accurate 3D maps, especially in textureless indoor environments.

Contribution

It is the first self-supervised approach for active stereo depth completion, combining SLAM-based landmarks and a new loss function for improved dense depth estimation.

Findings

Outperforms state-of-the-art on both real and synthetic datasets.

Produces more complete and safer 3D maps for robotic applications.

Effectively handles textureless indoor environments.

Abstract

Active stereo systems are used in many robotic applications that require 3D information. These depth sensors, however, suffer from stereo artefacts and do not provide dense depth estimates.In this work, we present the first self-supervised depth completion method for active stereo systems that predicts accurate dense depth maps. Our system leverages a feature-based visual inertial SLAM system to produce motion estimates and accurate (but sparse) 3D landmarks. The 3D landmarks are used both as model input and as supervision during training. The motion estimates are used in our novel reconstruction loss that relies on a combination of passive and active stereo frames, resulting in significant improvements in textureless areas that are common in indoor environments. Due to the nonexistence of publicly available active stereo datasets, we release a real dataset together with additional information for a publicly available synthetic dataset (TartanAir [42]) needed for active depth completion and prediction. Through rigorous evaluations we show that our method outperforms state of the art on both datasets. Additionally we show how our method obtains more complete, and therefore safer, 3D maps when used in a robotic platform.