Combining SLAM with muti-spectral photometric stereo for real-time dense 3D reconstruction

TL;DR

This paper introduces a real-time dense 3D reconstruction method combining semi-dense SLAM and multispectral photometric stereo, achieving detailed 3D models from monocular multispectral videos with low computational cost.

Contribution

It presents a novel framework that integrates semi-dense SLAM with multispectral photometric stereo for improved dense 3D reconstruction from monocular videos.

Findings

Effective dense 3D reconstructions with subtle textures

Improved accuracy over existing methods

Real-time performance demonstrated

Abstract

Obtaining dense 3D reconstrution with low computational cost is one of the important goals in the field of SLAM. In this paper we propose a dense 3D reconstruction framework from monocular multispectral video sequences using jointly semi-dense SLAM and Multispectral Photometric Stereo approaches. Starting from multispectral video, SALM (a) reconstructs a semi-dense 3D shape that will be densified;(b) recovers relative sparse depth map that is then fed as prioris into optimization-based multispectral photometric stereo for a more accurate dense surface normal recovery;(c)obtains camera pose that is subsequently used for conversion of view in the process of fusion where we combine the relative sparse point cloud with the dense surface normal using the automated cross-scale fusion method proposed in this paper to get a dense point cloud with subtle texture information. Experiments show that our method can effectively obtain denser 3D reconstructions.