High-Quality RGB-D Reconstruction via Multi-View Uncalibrated Photometric Stereo and Gradient-SDF

TL;DR

This paper introduces a novel multi-view RGB-D reconstruction method using gradient-SDF that jointly estimates camera pose, lighting, albedo, and surface normals, achieving more detailed and accurate surface geometry than existing methods.

Contribution

It presents a new approach that optimizes surface quantities directly on the volumetric representation using physically-based models, improving reconstruction fidelity.

Findings

Outperforms state-of-the-art in synthetic and real-world datasets

Accurately recovers camera poses and detailed surface geometry

Validates effectiveness with natural and point light source models

Abstract

Fine-detailed reconstructions are in high demand in many applications. However, most of the existing RGB-D reconstruction methods rely on pre-calculated accurate camera poses to recover the detailed surface geometry, where the representation of a surface needs to be adapted when optimizing different quantities. In this paper, we present a novel multi-view RGB-D based reconstruction method that tackles camera pose, lighting, albedo, and surface normal estimation via the utilization of a gradient signed distance field (gradient-SDF). The proposed method formulates the image rendering process using specific physically-based model(s) and optimizes the surface's quantities on the actual surface using its volumetric representation, as opposed to other works which estimate surface quantities only near the actual surface. To validate our method, we investigate two physically-based image formation models for natural light and point light source applications. The experimental results on synthetic and real-world datasets demonstrate that the proposed method can recover high-quality geometry of the surface more faithfully than the state-of-the-art and further improves the accuracy of estimated camera poses.