RobustMVS: Single Domain Generalized Deep Multi-view Stereo

TL;DR

RobustMVS introduces a new framework for multi-view stereo that enhances domain generalization by maintaining feature consistency across views, using a DepthClustering-guided Whitening loss, and is validated on a novel benchmark.

Contribution

The paper proposes RobustMVS, a novel MVS framework with a DepthClustering-guided Whitening loss for improved domain generalization in multi-view stereo.

Findings

Achieves superior performance on the new domain generalization benchmark.

Effectively maintains feature consistency across views.

Outperforms existing methods in unseen domain scenarios.

Abstract

Despite the impressive performance of Multi-view Stereo (MVS) approaches given plenty of training samples, the performance degradation when generalizing to unseen domains has not been clearly explored yet. In this work, we focus on the domain generalization problem in MVS. To evaluate the generalization results, we build a novel MVS domain generalization benchmark including synthetic and real-world datasets. In contrast to conventional domain generalization benchmarks, we consider a more realistic but challenging scenario, where only one source domain is available for training. The MVS problem can be analogized back to the feature matching task, and maintaining robust feature consistency among views is an important factor for improving generalization performance. To address the domain generalization problem in MVS, we propose a novel MVS framework, namely RobustMVS. A DepthClustering-guided Whitening (DCW) loss is further introduced to preserve the feature consistency among different views, which decorrelates multi-view features from viewpoint-specific style information based on geometric priors from depth maps. The experimental results further show that our method achieves superior performance on the domain generalization benchmark.