Multi-view Monocular Depth and Uncertainty Prediction with Deep SfM in Dynamic Environments

TL;DR

This paper enhances monocular depth and motion reconstruction in dynamic scenes by integrating learned uncertainty into a deep multi-view framework, improving accuracy and error detection.

Contribution

It introduces DeepV2cD, extending DeepV2D with learned uncertainty to improve depth estimation and error detection in dynamic environments.

Findings

DeepV2cD performs on par or better than state-of-the-art methods.

Uncertainty-based filtering reduces systematic errors.

Results show cleaner reconstructions in static and dynamic scene parts.

Abstract

3D reconstruction of depth and motion from monocular video in dynamic environments is a highly ill-posed problem due to scale ambiguities when projecting to the 2D image domain. In this work, we investigate the performance of the current State-of-the-Art (SotA) deep multi-view systems in such environments. We find that current supervised methods work surprisingly well despite not modelling individual object motions, but make systematic errors due to a lack of dense ground truth data. To detect such errors during usage, we extend the cost volume based Deep Video to Depth (DeepV2D) framework \cite{teed2018deepv2d} with a learned uncertainty. Our Deep Video to certain Depth (DeepV2cD) model allows i) to perform en par or better with current SotA and ii) achieve a better uncertainty measure than the naive Shannon entropy. Our experiments show that a simple filter strategy based on the uncertainty can significantly reduce systematic errors. This results in cleaner reconstructions both on static and dynamic parts of the scene.