Spurfies: Sparse Surface Reconstruction using Local Geometry Priors

TL;DR

Spurfies is a new sparse-view surface reconstruction method that leverages local geometry priors trained on synthetic data, significantly improving surface quality in limited-view scenarios and applicable to large scenes.

Contribution

We propose a neural point-based approach that disentangles geometry and appearance, enabling effective sparse-view reconstruction with synthetic-trained priors, outperforming existing methods.

Findings

Outperforms previous state-of-the-art by 35% in surface quality.

Effective on both bounded and unbounded scenes like Mip-NeRF 360.

Utilizes synthetic data to train local geometry priors for sparse-view reconstruction.

Abstract

We introduce Spurfies, a novel method for sparse-view surface reconstruction that disentangles appearance and geometry information to utilize local geometry priors trained on synthetic data. Recent research heavily focuses on 3D reconstruction using dense multi-view setups, typically requiring hundreds of images. However, these methods often struggle with few-view scenarios. Existing sparse-view reconstruction techniques often rely on multi-view stereo networks that need to learn joint priors for geometry and appearance from a large amount of data. In contrast, we introduce a neural point representation that disentangles geometry and appearance to train a local geometry prior using a subset of the synthetic ShapeNet dataset only. During inference, we utilize this surface prior as additional constraint for surface and appearance reconstruction from sparse input views via differentiable volume rendering, restricting the space of possible solutions. We validate the effectiveness of our method on the DTU dataset and demonstrate that it outperforms previous state of the art by 35% in surface quality while achieving competitive novel view synthesis quality. Moreover, in contrast to previous works, our method can be applied to larger, unbounded scenes, such as Mip-NeRF 360.