Unsupervised Monocular Depth Learning in Dynamic Scenes

TL;DR

This paper introduces a novel unsupervised method for monocular depth estimation in dynamic scenes by jointly learning depth, ego-motion, and object translation fields using photometric consistency and prior regularization, outperforming previous methods.

Contribution

It proposes a new unsupervised approach that leverages prior knowledge about 3D translation sparsity and rigidity to improve depth prediction in dynamic scenes without semantic input.

Findings

Achieves higher accuracy than prior methods on dynamic scenes

Uses only monocular images and photometric consistency for supervision

Regularization based on sparsity and rigidity is effective

Abstract

We present a method for jointly training the estimation of depth, ego-motion, and a dense 3D translation field of objects relative to the scene, with monocular photometric consistency being the sole source of supervision. We show that this apparently heavily underdetermined problem can be regularized by imposing the following prior knowledge about 3D translation fields: they are sparse, since most of the scene is static, and they tend to be constant for rigid moving objects. We show that this regularization alone is sufficient to train monocular depth prediction models that exceed the accuracy achieved in prior work for dynamic scenes, including methods that require semantic input. Code is at https://github.com/google-research/google-research/tree/master/depth_and_motion_learning .