Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting

TL;DR

This paper presents a novel 3D reconstruction method that jointly optimizes geometry, textures, camera poses, and lighting to produce highly detailed and consistent 3D models from consumer RGB-D sensors.

Contribution

It introduces a joint surface reconstruction approach combining Shape-from-Shading and spatially-varying lighting estimation for improved detail and texture accuracy.

Findings

Significantly enhances geometric detail in reconstructions.

Improves surface texture consistency.

Demonstrates effectiveness through extensive evaluations.

Abstract

We introduce a novel method to obtain high-quality 3D reconstructions from consumer RGB-D sensors. Our core idea is to simultaneously optimize for geometry encoded in a signed distance field (SDF), textures from automatically-selected keyframes, and their camera poses along with material and scene lighting. To this end, we propose a joint surface reconstruction approach that is based on Shape-from-Shading (SfS) techniques and utilizes the estimation of spatially-varying spherical harmonics (SVSH) from subvolumes of the reconstructed scene. Through extensive examples and evaluations, we demonstrate that our method dramatically increases the level of detail in the reconstructed scene geometry and contributes highly to consistent surface texture recovery.