Unsupervised Scale-consistent Depth Learning from Video

TL;DR

This paper introduces SC-Depth, a monocular depth estimation method trained on unlabelled videos that achieves scale-consistent predictions and improves SLAM robustness.

Contribution

It presents a geometry consistency loss and a self-discovered mask to enhance depth estimation from videos, enabling integration with SLAM systems.

Findings

High-quality depth results on KITTI and NYUv2 datasets

Improved SLAM robustness with scale-consistent depth

Effective handling of moving objects during training

Abstract

We propose a monocular depth estimator SC-Depth, which requires only unlabelled videos for training and enables the scale-consistent prediction at inference time. Our contributions include: (i) we propose a geometry consistency loss, which penalizes the inconsistency of predicted depths between adjacent views; (ii) we propose a self-discovered mask to automatically localize moving objects that violate the underlying static scene assumption and cause noisy signals during training; (iii) we demonstrate the efficacy of each component with a detailed ablation study and show high-quality depth estimation results in both KITTI and NYUv2 datasets. Moreover, thanks to the capability of scale-consistent prediction, we show that our monocular-trained deep networks are readily integrated into the ORB-SLAM2 system for more robust and accurate tracking. The proposed hybrid Pseudo-RGBD SLAM shows compelling results in KITTI, and it generalizes well to the KAIST dataset without additional training. Finally, we provide several demos for qualitative evaluation.