Depth Field Networks for Generalizable Multi-view Scene Representation

TL;DR

This paper introduces Depth Field Networks (DeFiNe), a novel approach that learns implicit, multi-view consistent scene representations using data augmentation and auxiliary view synthesis, achieving state-of-the-art depth estimation and strong domain generalization.

Contribution

It proposes a new method combining multi-view consistency, data augmentation, and auxiliary view synthesis to improve depth estimation and domain generalization without explicit geometric constraints.

Findings

Achieves state-of-the-art stereo and video depth estimation results.

Significantly improves zero-shot domain generalization.

Outperforms existing methods without explicit geometric constraints.

Abstract

Modern 3D computer vision leverages learning to boost geometric reasoning, mapping image data to classical structures such as cost volumes or epipolar constraints to improve matching. These architectures are specialized according to the particular problem, and thus require significant task-specific tuning, often leading to poor domain generalization performance. Recently, generalist Transformer architectures have achieved impressive results in tasks such as optical flow and depth estimation by encoding geometric priors as inputs rather than as enforced constraints. In this paper, we extend this idea and propose to learn an implicit, multi-view consistent scene representation, introducing a series of 3D data augmentation techniques as a geometric inductive prior to increase view diversity. We also show that introducing view synthesis as an auxiliary task further improves depth estimation. Our Depth Field Networks (DeFiNe) achieve state-of-the-art results in stereo and video depth estimation without explicit geometric constraints, and improve on zero-shot domain generalization by a wide margin.