SelfDeco: Self-Supervised Monocular Depth Completion in Challenging Indoor Environments

TL;DR

This paper introduces SelfDeco, a self-supervised neural network for monocular depth completion in complex indoor environments, effectively handling textureless, glossy, transparent, and dynamic scenes without dense labels.

Contribution

The novel architecture combines sparse convolution and pixel-adaptive convolutions, enabling accurate depth completion using only sparse depth and monocular video data.

Findings

Achieves 5-34% RMSE reduction on indoor datasets

Effective in challenging indoor scenarios with non-Lambertian surfaces

Outperforms existing methods in depth accuracy

Abstract

We present a novel algorithm for self-supervised monocular depth completion. Our approach is based on training a neural network that requires only sparse depth measurements and corresponding monocular video sequences without dense depth labels. Our self-supervised algorithm is designed for challenging indoor environments with textureless regions, glossy and transparent surface, non-Lambertian surfaces, moving people, longer and diverse depth ranges and scenes captured by complex ego-motions. Our novel architecture leverages both deep stacks of sparse convolution blocks to extract sparse depth features and pixel-adaptive convolutions to fuse image and depth features. We compare with existing approaches in NYUv2, KITTI, and NAVERLABS indoor datasets, and observe 5-34 % improvements in root-means-square error (RMSE) reduction.