ERF: Explicit Radiance Field Reconstruction From Scratch

TL;DR

This paper introduces an explicit 3D reconstruction method using hierarchical volumetric fields and inverse differentiable rendering, achieving high-quality, editable scene models from images without relying on implicit neural representations.

Contribution

The novel explicit volumetric scene representation and optimization approach enable high-quality, editable 3D reconstructions comparable to implicit methods, without sequential pipelines or controlled setups.

Findings

Reconstructed models are comparable to state-of-the-art implicit methods.

The method handles diverse scenes, including outdoor plants and furry toys.

Explicit models are more versatile and easier to edit.

Abstract

We propose a novel explicit dense 3D reconstruction approach that processes a set of images of a scene with sensor poses and calibrations and estimates a photo-real digital model. One of the key innovations is that the underlying volumetric representation is completely explicit in contrast to neural network-based (implicit) alternatives. We encode scenes explicitly using clear and understandable mappings of optimization variables to scene geometry and their outgoing surface radiance. We represent them using hierarchical volumetric fields stored in a sparse voxel octree. Robustly reconstructing such a volumetric scene model with millions of unknown variables from registered scene images only is a highly non-convex and complex optimization problem. To this end, we employ stochastic gradient descent (Adam) which is steered by an inverse differentiable renderer. We demonstrate that our method can reconstruct models of high quality that are comparable to state-of-the-art implicit methods. Importantly, we do not use a sequential reconstruction pipeline where individual steps suffer from incomplete or unreliable information from previous stages, but start our optimizations from uniformed initial solutions with scene geometry and radiance that is far off from the ground truth. We show that our method is general and practical. It does not require a highly controlled lab setup for capturing, but allows for reconstructing scenes with a vast variety of objects, including challenging ones, such as outdoor plants or furry toys. Finally, our reconstructed scene models are versatile thanks to their explicit design. They can be edited interactively which is computationally too costly for implicit alternatives.