Multiview Stereo with Cascaded Epipolar RAFT

TL;DR

CER-MVS introduces a novel multiview stereo approach based on RAFT, utilizing epipolar geometry and cascaded cost volumes to improve 3D reconstruction accuracy and completeness, achieving state-of-the-art results on major benchmarks.

Contribution

It presents CER-MVS, a new multiview stereo method that updates disparity fields with epipolar cost volumes and multiview fusion, differing from traditional 3D cost volume approaches.

Findings

Achieves second-best results on DTU dataset.

Sets new state-of-the-art on Tanks-and-Temples benchmark.

Demonstrates effective balance of completeness and accuracy in reconstructions.

Abstract

We address multiview stereo (MVS), an important 3D vision task that reconstructs a 3D model such as a dense point cloud from multiple calibrated images. We propose CER-MVS (Cascaded Epipolar RAFT Multiview Stereo), a new approach based on the RAFT (Recurrent All-Pairs Field Transforms) architecture developed for optical flow. CER-MVS introduces five new changes to RAFT: epipolar cost volumes, cost volume cascading, multiview fusion of cost volumes, dynamic supervision, and multiresolution fusion of depth maps. CER-MVS is significantly different from prior work in multiview stereo. Unlike prior work, which operates by updating a 3D cost volume, CER-MVS operates by updating a disparity field. Furthermore, we propose an adaptive thresholding method to balance the completeness and accuracy of the reconstructed point clouds. Experiments show that our approach achieves competitive performance on DTU (the second best among known results) and state-of-the-art performance on the Tanks-and-Temples benchmark (both the intermediate and advanced set). Code is available at https://github.com/princeton-vl/CER-MVS