Neural 3D Reconstruction in the Wild

TL;DR

This paper introduces a neural 3D reconstruction method capable of handling in-the-wild scenes with varying illumination, using a hybrid sampling technique and a new benchmark for evaluation.

Contribution

It presents a novel hybrid sampling approach and a benchmark for neural 3D reconstruction from internet photos in unconstrained environments.

Findings

Outperforms classical and neural methods on multiple metrics

Efficient surface reconstruction in diverse lighting conditions

New benchmark for in-the-wild 3D reconstruction

Abstract

We are witnessing an explosion of neural implicit representations in computer vision and graphics. Their applicability has recently expanded beyond tasks such as shape generation and image-based rendering to the fundamental problem of image-based 3D reconstruction. However, existing methods typically assume constrained 3D environments with constant illumination captured by a small set of roughly uniformly distributed cameras. We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections in the presence of varying illumination. To achieve this, we propose a hybrid voxel- and surface-guided sampling technique that allows for more efficient ray sampling around surfaces and leads to significant improvements in reconstruction quality. Further, we present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes. We perform extensive experiments, demonstrating that our approach surpasses both classical and neural reconstruction methods on a wide variety of metrics.