SparseCraft: Few-Shot Neural Reconstruction through Stereopsis Guided Geometric Linearization

TL;DR

SparseCraft is a fast, neural 3D reconstruction method that uses a novel shape learning strategy to produce high-quality results from very few images without pretrained models.

Contribution

It introduces a new implicit shape learning approach that enhances robustness and achieves state-of-the-art results in sparse-view 3D reconstruction and view synthesis.

Findings

Achieves state-of-the-art performance in sparse-view reconstruction

Requires less than 10 minutes for training

Does not rely on pretrained priors

Abstract

We present a novel approach for recovering 3D shape and view dependent appearance from a few colored images, enabling efficient 3D reconstruction and novel view synthesis. Our method learns an implicit neural representation in the form of a Signed Distance Function (SDF) and a radiance field. The model is trained progressively through ray marching enabled volumetric rendering, and regularized with learning-free multi-view stereo (MVS) cues. Key to our contribution is a novel implicit neural shape function learning strategy that encourages our SDF field to be as linear as possible near the level-set, hence robustifying the training against noise emanating from the supervision and regularization signals. Without using any pretrained priors, our method, called SparseCraft, achieves state-of-the-art performances both in novel-view synthesis and reconstruction from sparse views in standard benchmarks, while requiring less than 10 minutes for training.