DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data

TL;DR

DiverseDepth introduces a monocular depth estimation method that predicts affine-invariant depth, enabling accurate 3D shape reconstruction across diverse scenes with strong generalization, leveraging a new large-scale dataset and multi-curriculum training.

Contribution

The paper presents a novel dataset and a depth prediction method that generalizes well across diverse scenes by predicting affine-invariant depth, unlike previous metric or relative depth approaches.

Findings

Outperforms previous methods on 8 datasets in zero-shot tests

Reconstructs high-quality 3D shapes from predicted depth

Demonstrates strong generalization to diverse scenes

Abstract

We present a method for depth estimation with monocular images, which can predict high-quality depth on diverse scenes up to an affine transformation, thus preserving accurate shapes of a scene. Previous methods that predict metric depth often work well only for a specific scene. In contrast, learning relative depth (information of being closer or further) can enjoy better generalization, with the price of failing to recover the accurate geometric shape of the scene. In this work, we propose a dataset and methods to tackle this dilemma, aiming to predict accurate depth up to an affine transformation with good generalization to diverse scenes. First we construct a large-scale and diverse dataset, termed Diverse Scene Depth dataset (DiverseDepth), which has a broad range of scenes and foreground contents. Compared with previous learning objectives, i.e., learning metric depth or relative depth, we propose to learn the affine-invariant depth using our diverse dataset to ensure both generalization and high-quality geometric shapes of scenes. Furthermore, in order to train the model on the complex dataset effectively, we propose a multi-curriculum learning method. Experiments show that our method outperforms previous methods on 8 datasets by a large margin with the zero-shot test setting, demonstrating the excellent generalization capacity of the learned model to diverse scenes. The reconstructed point clouds with the predicted depth show that our method can recover high-quality 3D shapes. Code and dataset are available at: https://tinyurl.com/DiverseDepth