DynOcc: Learning Single-View Depth from Dynamic Occlusion Cues

TL;DR

DynOcc introduces a large-scale in-the-wild dynamic scene dataset for single-view depth estimation, leveraging occlusion cues to improve depth accuracy and boundary sharpness, advancing the field beyond static or domain-limited datasets.

Contribution

This work presents the first dynamic in-the-wild depth dataset, DynOcc, and a novel occlusion-based approach for depth inference from video frames, achieving state-of-the-art results.

Findings

Achieved state-of-the-art WHDR scores.

Created a dataset with 22 million depth pairs from 91,000 frames.

Depth maps trained on DynOcc preserve sharper boundaries.

Abstract

Recently, significant progress has been made in single-view depth estimation thanks to increasingly large and diverse depth datasets. However, these datasets are largely limited to specific application domains (e.g. indoor, autonomous driving) or static in-the-wild scenes due to hardware constraints or technical limitations of 3D reconstruction. In this paper, we introduce the first depth dataset DynOcc consisting of dynamic in-the-wild scenes. Our approach leverages the occlusion cues in these dynamic scenes to infer depth relationships between points of selected video frames. To achieve accurate occlusion detection and depth order estimation, we employ a novel occlusion boundary detection, filtering and thinning scheme followed by a robust foreground/background classification method. In total our DynOcc dataset contains 22M depth pairs out of 91K frames from a diverse set of videos. Using our dataset we achieved state-of-the-art results measured in weighted human disagreement rate (WHDR). We also show that the inferred depth maps trained with DynOcc can preserve sharper depth boundaries.