DeepMVS: Learning Multi-view Stereopsis

TL;DR

DeepMVS introduces a deep learning approach for multi-view stereo reconstruction that leverages synthetic pretraining, multi-image aggregation, and multi-layer features to produce high-quality disparity maps, outperforming traditional methods.

Contribution

The paper presents a novel deep ConvNet architecture for multi-view stereo that effectively aggregates unordered images and utilizes pretraining on synthetic data for improved accuracy.

Findings

Outperforms state-of-the-art MVS algorithms on ETH3D Benchmark.

Excels in reconstructing near-textureless regions and thin structures.

Demonstrates the effectiveness of synthetic pretraining and multi-layer feature integration.

Abstract

We present DeepMVS, a deep convolutional neural network (ConvNet) for multi-view stereo reconstruction. Taking an arbitrary number of posed images as input, we first produce a set of plane-sweep volumes and use the proposed DeepMVS network to predict high-quality disparity maps. The key contributions that enable these results are (1) supervised pretraining on a photorealistic synthetic dataset, (2) an effective method for aggregating information across a set of unordered images, and (3) integrating multi-layer feature activations from the pre-trained VGG-19 network. We validate the efficacy of DeepMVS using the ETH3D Benchmark. Our results show that DeepMVS compares favorably against state-of-the-art conventional MVS algorithms and other ConvNet based methods, particularly for near-textureless regions and thin structures.