Surface-Based Visibility-Guided Uncertainty for Continuous Active 3D Neural Reconstruction

TL;DR

This paper introduces a surface-based visibility field (SBV) that estimates visibility-guided uncertainty during continuous active 3D neural reconstruction, improving view selection and reconstruction quality.

Contribution

It presents a novel SBV method that estimates visibility-guided uncertainty during ongoing learning, enabling more accurate view selection in continuous active 3D reconstruction.

Findings

SBV-guided view selection improves reconstruction performance by up to 11.6%.

The method effectively captures uncertainties across all surface regions.

Experiments on multiple datasets demonstrate robustness and accuracy.

Abstract

View selection is critical in active 3D neural reconstruction as it impacts the contents of training set and resulting final output quality. Recent view selection strategies emphasize the visibility when evaluating model uncertainty in active 3D reconstruction. However, existing approaches estimate visibility only after the model fully converges, which has confined their application primarily to non-continuous active learning settings. This paper proposes Surface-Based Visibility field (SBV) that successfully estimates the visibility-guided uncertainty in continuous active 3D neural reconstruction. During learning neural implicit surfaces, our model learns rendering uncertainties and infers surface confidence values derived from signed distance functions. It then updates surface confidences using a voxel grid, robustly deducing the surface-based visibility for uncertainties. This approach captures uncertainties across all regions, whether well-defined surfaces or ambiguous areas, ensuring accurate visibility measurement in continuous active learning. Experiments on benchmark datasets-Tanks and Temples, BlendedMVS, Blender, DTU-and the newly proposed imbalanced viewpoint dataset (ImBView) show that view selection based on SBV-guided uncertainty improves performance by up to 11.6% over existing methods, highlighting its effectiveness in challenging reconstruction scenarios.