High-Fidelity and Generalizable Neural Surface Reconstruction with Sparse Feature Volumes

TL;DR

This paper introduces a sparse feature volume approach for neural surface reconstruction that significantly improves resolution and efficiency, enabling high-quality reconstructions with less memory and on standard hardware.

Contribution

The authors propose a novel sparse volume method with custom algorithms, allowing higher resolution reconstructions without performance loss, surpassing dense volume limitations.

Findings

Reduces storage by over 50 times

Enables $512^3$ resolution on standard hardware

Achieves superior reconstruction accuracy

Abstract

Generalizable neural surface reconstruction has become a compelling technique to reconstruct from few images without per-scene optimization, where dense 3D feature volume has proven effective as a global representation of scenes. However, the dense representation does not scale well to increasing voxel resolutions, severely limiting the reconstruction quality. We thus present a sparse representation method, that maximizes memory efficiency and enables significantly higher resolution reconstructions on standard hardware. We implement this through a two-stage approach: First training a network to predict voxel occupancies from posed images and associated depth maps, then computing features and performing volume rendering only in voxels with sufficiently high occupancy estimates. To support this sparse representation, we developed custom algorithms for efficient sampling, feature aggregation, and querying from sparse volumes-overcoming the dense-volume assumptions inherent in existing works. Experiments on public datasets demonstrate that our approach reduces storage requirements by more than 50 times without performance degradation, enabling reconstructions at $512^3$ resolution compared to the typical $128^3$ on similar hardware, and achieving superior reconstruction accuracy over current state-of-the-art methods.