SG-NeRF: Neural Surface Reconstruction with Scene Graph Optimization

TL;DR

SG-NeRF introduces a scene graph-based optimization method for neural surface reconstruction that effectively handles noisy camera poses, improving 3D modeling quality and robustness in real-world scenarios.

Contribution

The paper presents a novel scene graph optimization framework for NeRFs that mitigates the impact of outlier camera poses during 3D surface reconstruction.

Findings

Demonstrates robustness to noisy camera poses in various datasets.

Outperforms existing methods in reconstruction quality.

Provides a new dataset with outlier poses for evaluation.

Abstract

3D surface reconstruction from images is essential for numerous applications. Recently, Neural Radiance Fields (NeRFs) have emerged as a promising framework for 3D modeling. However, NeRFs require accurate camera poses as input, and existing methods struggle to handle significantly noisy pose estimates (i.e., outliers), which are commonly encountered in real-world scenarios. To tackle this challenge, we present a novel approach that optimizes radiance fields with scene graphs to mitigate the influence of outlier poses. Our method incorporates an adaptive inlier-outlier confidence estimation scheme based on scene graphs, emphasizing images of high compatibility with the neighborhood and consistency in the rendering quality. We also introduce an effective intersection-over-union (IoU) loss to optimize the camera pose and surface geometry, together with a coarse-to-fine strategy to facilitate the training. Furthermore, we propose a new dataset containing typical outlier poses for a detailed evaluation. Experimental results on various datasets consistently demonstrate the effectiveness and superiority of our method over existing approaches, showcasing its robustness in handling outliers and producing high-quality 3D reconstructions. Our code and data are available at: \url{https://github.com/Iris-cyy/SG-NeRF}.