Unsupervised Monocular Depth Reconstruction of Non-Rigid Scenes

TL;DR

This paper introduces an unsupervised monocular depth estimation framework capable of reconstructing both rigid and non-rigid scene components without explicit camera motion modeling, effective on challenging dynamic videos.

Contribution

It presents a novel unsupervised method that jointly reconstructs rigid and non-rigid scene parts using dense correspondences and the as-rigid-as-possible hypothesis, without explicit camera motion modeling.

Findings

Effective depth reconstruction of non-rigid scenes from monocular videos

Joint depth estimation and motion segmentation capability

Promising results on challenging dynamic scene videos

Abstract

Monocular depth reconstruction of complex and dynamic scenes is a highly challenging problem. While for rigid scenes learning-based methods have been offering promising results even in unsupervised cases, there exists little to no literature addressing the same for dynamic and deformable scenes. In this work, we present an unsupervised monocular framework for dense depth estimation of dynamic scenes, which jointly reconstructs rigid and non-rigid parts without explicitly modelling the camera motion. Using dense correspondences, we derive a training objective that aims to opportunistically preserve pairwise distances between reconstructed 3D points. In this process, the dense depth map is learned implicitly using the as-rigid-as-possible hypothesis. Our method provides promising results, demonstrating its capability of reconstructing 3D from challenging videos of non-rigid scenes. Furthermore, the proposed method also provides unsupervised motion segmentation results as an auxiliary output.