IB-MVS: An Iterative Algorithm for Deep Multi-View Stereo based on Binary Decisions

TL;DR

IB-MVS introduces an iterative deep-learning approach for multi-view stereo that uses binary decisions to efficiently estimate high-resolution, precise depth maps without large cost volumes, achieving competitive results.

Contribution

The paper proposes a novel binary decision-based iterative algorithm for deep multi-view stereo, enabling efficient high-resolution depth estimation without large cost volumes.

Findings

Achieves high-resolution, precise depth maps efficiently.

Performs competitively on DTU, Tanks and Temples, ETH3D benchmarks.

Handles occlusions through learned pixelwise fusion.

Abstract

We present a novel deep-learning-based method for Multi-View Stereo. Our method estimates high resolution and highly precise depth maps iteratively, by traversing the continuous space of feasible depth values at each pixel in a binary decision fashion. The decision process leverages a deep-network architecture: this computes a pixelwise binary mask that establishes whether each pixel actual depth is in front or behind its current iteration individual depth hypothesis. Moreover, in order to handle occluded regions, at each iteration the results from different source images are fused using pixelwise weights estimated by a second network. Thanks to the adopted binary decision strategy, which permits an efficient exploration of the depth space, our method can handle high resolution images without trading resolution and precision. This sets it apart from most alternative learning-based Multi-View Stereo methods, where the explicit discretization of the depth space requires the processing of large cost volumes. We compare our method with state-of-the-art Multi-View Stereo methods on the DTU, Tanks and Temples and the challenging ETH3D benchmarks and show competitive results.