VolRecon: Volume Rendering of Signed Ray Distance Functions for Generalizable Multi-View Reconstruction

TL;DR

VolRecon introduces a generalizable neural implicit scene reconstruction method using Signed Ray Distance Functions, combining multi-view and volume features, and employing a ray transformer for detailed, noise-free 3D reconstructions with strong generalization.

Contribution

It proposes a novel SRDF-based implicit reconstruction approach that generalizes across scenes, outperforming existing methods in sparse view scenarios and maintaining accuracy in large-scale benchmarks.

Findings

Outperforms SparseNeuS by 30% in sparse view reconstruction

Achieves comparable accuracy to MVSNet in full view reconstruction

Demonstrates strong generalization on ETH3D benchmark

Abstract

The success of the Neural Radiance Fields (NeRF) in novel view synthesis has inspired researchers to propose neural implicit scene reconstruction. However, most existing neural implicit reconstruction methods optimize per-scene parameters and therefore lack generalizability to new scenes. We introduce VolRecon, a novel generalizable implicit reconstruction method with Signed Ray Distance Function (SRDF). To reconstruct the scene with fine details and little noise, VolRecon combines projection features aggregated from multi-view features, and volume features interpolated from a coarse global feature volume. Using a ray transformer, we compute SRDF values of sampled points on a ray and then render color and depth. On DTU dataset, VolRecon outperforms SparseNeuS by about 30% in sparse view reconstruction and achieves comparable accuracy as MVSNet in full view reconstruction. Furthermore, our approach exhibits good generalization performance on the large-scale ETH3D benchmark.