GoMVS: Geometrically Consistent Cost Aggregation for Multi-View Stereo

TL;DR

GoMVS introduces a geometrically consistent cost aggregation method for multi-view stereo that leverages surface normals to improve depth estimation accuracy, achieving state-of-the-art results.

Contribution

The paper proposes the GCP module that propagates costs based on local geometric smoothness, effectively handling geometric inconsistency in cost volumes.

Findings

Achieves new state-of-the-art performance on multiple datasets.

Ranks 1st on the Tanks & Temple Advanced benchmark.

Effectively handles geometric inconsistency in cost aggregation.

Abstract

Matching cost aggregation plays a fundamental role in learning-based multi-view stereo networks. However, directly aggregating adjacent costs can lead to suboptimal results due to local geometric inconsistency. Related methods either seek selective aggregation or improve aggregated depth in the 2D space, both are unable to handle geometric inconsistency in the cost volume effectively. In this paper, we propose GoMVS to aggregate geometrically consistent costs, yielding better utilization of adjacent geometries. More specifically, we correspond and propagate adjacent costs to the reference pixel by leveraging the local geometric smoothness in conjunction with surface normals. We achieve this by the geometric consistent propagation (GCP) module. It computes the correspondence from the adjacent depth hypothesis space to the reference depth space using surface normals, then uses the correspondence to propagate adjacent costs to the reference geometry, followed by a convolution for aggregation. Our method achieves new state-of-the-art performance on DTU, Tanks & Temple, and ETH3D datasets. Notably, our method ranks 1st on the Tanks & Temple Advanced benchmark.