DVP-MVS: Synergize Depth-Edge and Visibility Prior for Multi-View Stereo

TL;DR

DVP-MVS introduces a novel multi-view stereo approach that synergizes depth-edge alignment and visibility priors, enhancing robustness and accuracy in textureless and occluded regions through innovative patch deformation techniques.

Contribution

The paper proposes DVP-MVS, integrating depth-edge alignment and cross-view visibility priors to improve patch deformation in multi-view stereo, addressing edge-skipping and occlusion issues.

Findings

Achieves state-of-the-art results on ETH3D and Tanks & Temples benchmarks.

Demonstrates robustness in textureless and occluded areas.

Improves patch deformation accuracy through depth-edge and visibility integration.

Abstract

Patch deformation-based methods have recently exhibited substantial effectiveness in multi-view stereo, due to the incorporation of deformable and expandable perception to reconstruct textureless areas. However, such approaches typically focus on exploring correlative reliable pixels to alleviate match ambiguity during patch deformation, but ignore the deformation instability caused by mistaken edge-skipping and visibility occlusion, leading to potential estimation deviation. To remedy the above issues, we propose DVP-MVS, which innovatively synergizes depth-edge aligned and cross-view prior for robust and visibility-aware patch deformation. Specifically, to avoid unexpected edge-skipping, we first utilize Depth Anything V2 followed by the Roberts operator to initialize coarse depth and edge maps respectively, both of which are further aligned through an erosion-dilation strategy to generate fine-grained homogeneous boundaries for guiding patch deformation. In addition, we reform view selection weights as visibility maps and restore visible areas by cross-view depth reprojection, then regard them as cross-view prior to facilitate visibility-aware patch deformation. Finally, we improve propagation and refinement with multi-view geometry consistency by introducing aggregated visible hemispherical normals based on view selection and local projection depth differences based on epipolar lines, respectively. Extensive evaluations on ETH3D and Tanks & Temples benchmarks demonstrate that our method can achieve state-of-the-art performance with excellent robustness and generalization.