Perspective-aware fusion of incomplete depth maps and surface normals for accurate 3D reconstruction

TL;DR

This paper introduces a perspective-aware fusion method for incomplete depth and surface normal maps, improving the accuracy of 3D surface reconstruction from single-camera sensor data.

Contribution

It extends existing orthographic fusion techniques by explicitly modeling perspective projection and inpaints missing depth data using surface normals.

Findings

Improved 3D reconstruction accuracy demonstrated on DiLiGenT-MV dataset.

Effectiveness of perspective-aware fusion over orthographic methods.

Handling of missing depth data through normal-based inpainting.

Abstract

We address the problem of reconstructing 3D surfaces from depth and surface normal maps acquired by a sensor system based on a single perspective camera. Depth and normal maps can be obtained through techniques such as structured-light scanning and photometric stereo, respectively. We propose a perspective-aware log-depth fusion approach that extends existing orthographic gradient-based depth-normals fusion methods by explicitly accounting for perspective projection, leading to metrically accurate 3D reconstructions. Additionally, the method handles missing depth measurements by leveraging available surface normal information to inpaint gaps. Experiments on the DiLiGenT-MV data set demonstrate the effectiveness of our approach and highlight the importance of perspective-aware depth-normals fusion.