A Benchmark and a Baseline for Robust Multi-view Depth Estimation

TL;DR

This paper introduces a comprehensive benchmark for multi-view depth estimation across diverse datasets and proposes a robust baseline model with scale augmentation to improve generalization in various domains.

Contribution

The work presents a new benchmark for evaluating multi-view depth estimation methods across multiple datasets and domains, along with a novel baseline model employing scale augmentation for better robustness.

Findings

Imbalanced performance of recent methods across datasets

Current approaches do not generalize well when camera poses are known

The proposed baseline with scale augmentation improves robustness

Abstract

Recent deep learning approaches for multi-view depth estimation are employed either in a depth-from-video or a multi-view stereo setting. Despite different settings, these approaches are technically similar: they correlate multiple source views with a keyview to estimate a depth map for the keyview. In this work, we introduce the Robust Multi-View Depth Benchmark that is built upon a set of public datasets and allows evaluation in both settings on data from different domains. We evaluate recent approaches and find imbalanced performances across domains. Further, we consider a third setting, where camera poses are available and the objective is to estimate the corresponding depth maps with their correct scale. We show that recent approaches do not generalize across datasets in this setting. This is because their cost volume output runs out of distribution. To resolve this, we present the Robust MVD Baseline model for multi-view depth estimation, which is built upon existing components but employs a novel scale augmentation procedure. It can be applied for robust multi-view depth estimation, independent of the target data. We provide code for the proposed benchmark and baseline model at https://github.com/lmb-freiburg/robustmvd.