D$^3$epth: Self-Supervised Depth Estimation with Dynamic Mask in Dynamic Scenes

TL;DR

D$^3$epth introduces a self-supervised depth estimation method tailored for dynamic scenes, utilizing dynamic masks and uncertainty guidance to improve accuracy in environments with moving objects.

Contribution

It proposes a novel dynamic mask construction and spectral entropy-based uncertainty module for self-supervised depth estimation in dynamic scenes, addressing limitations of static scene assumptions.

Findings

Outperforms existing self-supervised methods on KITTI and Cityscapes datasets.

Effectively handles dynamic objects through dynamic masking strategies.

Improves depth estimation accuracy in dynamic environments.

Abstract

Depth estimation is a crucial technology in robotics. Recently, self-supervised depth estimation methods have demonstrated great potential as they can efficiently leverage large amounts of unlabelled real-world data. However, most existing methods are designed under the assumption of static scenes, which hinders their adaptability in dynamic environments. To address this issue, we present D$^3$epth, a novel method for self-supervised depth estimation in dynamic scenes. It tackles the challenge of dynamic objects from two key perspectives. First, within the self-supervised framework, we design a reprojection constraint to identify regions likely to contain dynamic objects, allowing the construction of a dynamic mask that mitigates their impact at the loss level. Second, for multi-frame depth estimation, we introduce a cost volume auto-masking strategy that leverages adjacent frames to identify regions associated with dynamic objects and generate corresponding masks. This provides guidance for subsequent processes. Furthermore, we propose a spectral entropy uncertainty module that incorporates spectral entropy to guide uncertainty estimation during depth fusion, effectively addressing issues arising from cost volume computation in dynamic environments. Extensive experiments on KITTI and Cityscapes datasets demonstrate that the proposed method consistently outperforms existing self-supervised monocular depth estimation baselines. Code is available at \url{https://github.com/Csyunling/D3epth}.