Geometric Prior-Guided Neural Implicit Surface Reconstruction in the Wild

TL;DR

This paper introduces a geometric prior-guided neural implicit surface reconstruction method that improves accuracy and detail in uncontrolled environments by integrating sparse 3D points and normal priors, outperforming existing techniques.

Contribution

The novel approach combines sparse SfM points and robust normal priors with surface optimization to enhance in-the-wild 3D reconstruction accuracy.

Findings

Achieves superior surface accuracy on Heritage-Recon benchmark.

Effectively handles uncontrolled environments with transient occlusions.

Produces high-quality reconstructions of cultural heritage landmarks.

Abstract

Neural implicit surface reconstruction using volume rendering techniques has recently achieved significant advancements in creating high-fidelity surfaces from multiple 2D images. However, current methods primarily target scenes with consistent illumination and struggle to accurately reconstruct 3D geometry in uncontrolled environments with transient occlusions or varying appearances. While some neural radiance field (NeRF)-based variants can better manage photometric variations and transient objects in complex scenes, they are designed for novel view synthesis rather than precise surface reconstruction due to limited surface constraints. To overcome this limitation, we introduce a novel approach that applies multiple geometric constraints to the implicit surface optimization process, enabling more accurate reconstructions from unconstrained image collections. First, we utilize sparse 3D points from structure-from-motion (SfM) to refine the signed distance function estimation for the reconstructed surface, with a displacement compensation to accommodate noise in the sparse points. Additionally, we employ robust normal priors derived from a normal predictor, enhanced by edge prior filtering and multi-view consistency constraints, to improve alignment with the actual surface geometry. Extensive testing on the Heritage-Recon benchmark and other datasets has shown that the proposed method can accurately reconstruct surfaces from in-the-wild images, yielding geometries with superior accuracy and granularity compared to existing techniques. Our approach enables high-quality 3D reconstruction of various landmarks, making it applicable to diverse scenarios such as digital preservation of cultural heritage sites.