SelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning

TL;DR

This paper introduces SelfTune, a self-supervised learning method that enables monocular depth estimation to produce metrically scaled depths by leveraging monocular SLAM for accurate camera pose information.

Contribution

We propose a novel teacher-student self-supervised framework that incorporates monocular SLAM to resolve scale ambiguity in monocular depth estimation.

Findings

Improves depth accuracy over recent methods on EuRoC, OpenLORIS, and ScanNet datasets.

Enables metrically scaled depth predictions in diverse environments.

Applicable to mobile robot navigation and real-world scenarios.

Abstract

Monocular depth estimation in the wild inherently predicts depth up to an unknown scale. To resolve scale ambiguity issue, we present a learning algorithm that leverages monocular simultaneous localization and mapping (SLAM) with proprioceptive sensors. Such monocular SLAM systems can provide metrically scaled camera poses. Given these metric poses and monocular sequences, we propose a self-supervised learning method for the pre-trained supervised monocular depth networks to enable metrically scaled depth estimation. Our approach is based on a teacher-student formulation which guides our network to predict high-quality depths. We demonstrate that our approach is useful for various applications such as mobile robot navigation and is applicable to diverse environments. Our full system shows improvements over recent self-supervised depth estimation and completion methods on EuRoC, OpenLORIS, and ScanNet datasets.