Neural shape reconstruction from multiple views with static pattern projection

TL;DR

This paper introduces a neural network-based method for 3D shape reconstruction that allows flexible camera and projector movement, auto-calibrating their poses during the process for improved usability.

Contribution

It presents a novel neural signed-distance-field approach with volumetric differential rendering enabling shape recovery from moving camera and projector without fixed calibration.

Findings

Effective 3D reconstruction on synthetic images

Successful real-world shape recovery

Auto-calibration of camera and projector poses

Abstract

Active-stereo-based 3D shape measurement is crucial for various purposes, such as industrial inspection, reverse engineering, and medical systems, due to its strong ability to accurately acquire the shape of textureless objects. Active stereo systems typically consist of a camera and a pattern projector, tightly fixed to each other, and precise calibration between a camera and a projector is required, which in turn decreases the usability of the system. If a camera and a projector can be freely moved during shape scanning process, it will drastically increase the convenience of the usability of the system. To realize it, we propose a technique to recover the shape of the target object by capturing multiple images while both the camera and the projector are in motion, and their relative poses are auto-calibrated by our neural signed-distance-field (NeuralSDF) using novel volumetric differential rendering technique. In the experiment, the proposed method is evaluated by performing 3D reconstruction using both synthetic and real images.