TSAR-MVS: Textureless-aware Segmentation and Correlative Refinement Guided Multi-View Stereo

TL;DR

TSAR-MVS introduces a novel multi-view stereo approach that effectively handles textureless regions through segmentation, correlation refinement, and filtering, resulting in improved 3D reconstruction accuracy.

Contribution

The paper presents a new textureless-aware segmentation and refinement framework that enhances multi-view stereo reconstruction, especially in challenging textureless areas, with strong experimental validation.

Findings

Outperforms existing methods on ETH3D, Tanks & Temples, Strecha datasets

Effectively reconstructs large textureless regions

Demonstrates strong generalization capability

Abstract

The reconstruction of textureless areas has long been a challenging problem in MVS due to lack of reliable pixel correspondences between images. In this paper, we propose the Textureless-aware Segmentation And Correlative Refinement guided Multi-View Stereo (TSAR-MVS), a novel method that effectively tackles challenges posed by textureless areas in 3D reconstruction through filtering, refinement and segmentation. First, we implement the joint hypothesis filtering, a technique that merges a confidence estimator with a disparity discontinuity detector to eliminate incorrect depth estimations. Second, to spread the pixels with confident depth, we introduce an iterative correlation refinement strategy that leverages RANSAC to generate 3D planes based on superpixels, succeeded by a weighted median filter for broadening the influence of accurately determined pixels. Finally, we present a textureless-aware segmentation method that leverages edge detection and line detection for accurately identify large textureless regions for further depth completion. Experiments on ETH3D, Tanks & Temples and Strecha datasets demonstrate the superior performance and strong generalization capability of our proposed method.