IA-MVS: Instance-Focused Adaptive Depth Sampling for Multi-View Stereo

TL;DR

The paper introduces IA-MVS, an instance-focused adaptive depth sampling method that improves multi-view stereo accuracy by narrowing depth hypotheses and refining on individual instances, achieving state-of-the-art results.

Contribution

It presents a novel instance-adaptive approach for depth sampling in MVS, including a filtering mechanism and confidence estimation model, without extra training overhead.

Findings

Achieves state-of-the-art performance on DTU benchmark.

Enhances depth estimation precision through instance-based refinement.

Improves robustness with intra-instance depth continuity priors.

Abstract

Multi-view stereo (MVS) models based on progressive depth hypothesis narrowing have made remarkable advancements. However, existing methods haven't fully utilized the potential that the depth coverage of individual instances is smaller than that of the entire scene, which restricts further improvements in depth estimation precision. Moreover, inevitable deviations in the initial stage accumulate as the process advances. In this paper, we propose Instance-Adaptive MVS (IA-MVS). It enhances the precision of depth estimation by narrowing the depth hypothesis range and conducting refinement on each instance. Additionally, a filtering mechanism based on intra-instance depth continuity priors is incorporated to boost robustness. Furthermore, recognizing that existing confidence estimation can degrade IA-MVS performance on point clouds. We have developed a detailed mathematical model for confidence estimation based on conditional probability. The proposed method can be widely applied in models based on MVSNet without imposing extra training burdens. Our method achieves state-of-the-art performance on the DTU benchmark. The source code is available at https://github.com/KevinWang73106/IA-MVS.