Neural Active Structure-from-Motion in Dark and Textureless Environment

TL;DR

This paper introduces a novel Active SfM method that jointly reconstructs 3D shape and estimates pose using neural signed distance fields from images with projected patterns, effective in dark and textureless environments.

Contribution

It presents a full optimization framework combining shape reconstruction and pose estimation using Neural-SDF for structured light systems in textureless scenes.

Findings

Accurate shape reconstruction achieved in textureless environments.

Effective pose estimation from pattern-only images.

Method outperforms traditional feature-based techniques.

Abstract

Active 3D measurement, especially structured light (SL) has been widely used in various fields for its robustness against textureless or equivalent surfaces by low light illumination. In addition, reconstruction of large scenes by moving the SL system has become popular, however, there have been few practical techniques to obtain the system's precise pose information only from images, since most conventional techniques are based on image features, which cannot be retrieved under textureless environments. In this paper, we propose a simultaneous shape reconstruction and pose estimation technique for SL systems from an image set where sparsely projected patterns onto the scene are observed (i.e. no scene texture information), which we call Active SfM. To achieve this, we propose a full optimization framework of the volumetric shape that employs neural signed distance fields (Neural-SDF) for SL with the goal of not only reconstructing the scene shape but also estimating the poses for each motion of the system. Experimental results show that the proposed method is able to achieve accurate shape reconstruction as well as pose estimation from images where only projected patterns are observed.