SC-NeuS: Consistent Neural Surface Reconstruction from Sparse and Noisy Views

TL;DR

SC-NeuS introduces a novel end-to-end method for neural surface reconstruction that effectively handles sparse and noisy views by leveraging multi-view geometric constraints and a differentiable on-surface intersection, improving detail and consistency.

Contribution

The paper presents a new approach that jointly learns neural surfaces and refines camera poses using explicit multi-view geometric constraints, unlike previous methods.

Findings

Achieves better surface reconstruction with fine details from sparse, noisy views.

Outperforms previous state-of-the-art methods on public datasets.

Provides a fast differentiable on-surface intersection for multi-view constraints.

Abstract

The recent neural surface reconstruction by volume rendering approaches have made much progress by achieving impressive surface reconstruction quality, but are still limited to dense and highly accurate posed views. To overcome such drawbacks, this paper pays special attention on the consistent surface reconstruction from sparse views with noisy camera poses. Unlike previous approaches, the key difference of this paper is to exploit the multi-view constraints directly from the explicit geometry of the neural surface, which can be used as effective regularization to jointly learn the neural surface and refine the camera poses. To build effective multi-view constraints, we introduce a fast differentiable on-surface intersection to generate on-surface points, and propose view-consistent losses based on such differentiable points to regularize the neural surface learning. Based on this point, we propose a jointly learning strategy for neural surface and camera poses, named SC-NeuS, to perform geometry-consistent surface reconstruction in an end-to-end manner. With extensive evaluation on public datasets, our SC-NeuS can achieve consistently better surface reconstruction results with fine-grained details than previous state-of-the-art neural surface reconstruction approaches, especially from sparse and noisy camera views.