Detail-preserving and Content-aware Variational Multi-view Stereo Reconstruction

TL;DR

This paper introduces a novel multi-view stereo reconstruction method that effectively preserves fine details and sharp features while reducing noise, outperforming existing techniques on benchmark datasets.

Contribution

The proposed DCV method combines a new inter-image similarity measure with adaptive mesh denoising, advancing detail preservation and noise suppression in 3D reconstruction.

Findings

Outperforms state-of-the-art methods on benchmark datasets

Achieves higher accuracy and completeness in reconstructions

Effectively preserves fine-scale details and sharp features

Abstract

Accurate recovery of 3D geometrical surfaces from calibrated 2D multi-view images is a fundamental yet active research area in computer vision. Despite the steady progress in multi-view stereo reconstruction, most existing methods are still limited in recovering fine-scale details and sharp features while suppressing noises, and may fail in reconstructing regions with few textures. To address these limitations, this paper presents a Detail-preserving and Content-aware Variational (DCV) multi-view stereo method, which reconstructs the 3D surface by alternating between reprojection error minimization and mesh denoising. In reprojection error minimization, we propose a novel inter-image similarity measure, which is effective to preserve fine-scale details of the reconstructed surface and builds a connection between guided image filtering and image registration. In mesh denoising, we propose a content-aware $\ell_{p}$-minimization algorithm by adaptively estimating the $p$ value and regularization parameters based on the current input. It is much more promising in suppressing noise while preserving sharp features than conventional isotropic mesh smoothing. Experimental results on benchmark datasets demonstrate that our DCV method is capable of recovering more surface details, and obtains cleaner and more accurate reconstructions than state-of-the-art methods. In particular, our method achieves the best results among all published methods on the Middlebury dino ring and dino sparse ring datasets in terms of both completeness and accuracy.