Curvature-guided dynamic scale networks for Multi-view Stereo

TL;DR

This paper introduces CDSFNet, a dynamic scale feature extraction network guided by surface curvature, which improves multi-view stereo depth estimation accuracy and efficiency, especially in complex outdoor scenes.

Contribution

It proposes a novel curvature-guided dynamic scale convolutional network for robust feature extraction in MVS, enhancing accuracy without heavy computation.

Findings

Outperforms state-of-the-art methods on outdoor scenes

Improves reconstruction completeness

Enables higher resolution processing with faster runtime

Abstract

Multi-view stereo (MVS) is a crucial task for precise 3D reconstruction. Most recent studies tried to improve the performance of matching cost volume in MVS by designing aggregated 3D cost volumes and their regularization. This paper focuses on learning a robust feature extraction network to enhance the performance of matching costs without heavy computation in the other steps. In particular, we present a dynamic scale feature extraction network, namely, CDSFNet. It is composed of multiple novel convolution layers, each of which can select a proper patch scale for each pixel guided by the normal curvature of the image surface. As a result, CDFSNet can estimate the optimal patch scales to learn discriminative features for accurate matching computation between reference and source images. By combining the robust extracted features with an appropriate cost formulation strategy, our resulting MVS architecture can estimate depth maps more precisely. Extensive experiments showed that the proposed method outperforms other state-of-the-art methods on complex outdoor scenes. It significantly improves the completeness of reconstructed models. As a result, the method can process higher resolution inputs within faster run-time and lower memory than other MVS methods. Our source code is available at url{https://github.com/TruongKhang/cds-mvsnet}.