Single-shot fringe projection profilometry based on Deep Learning and Computer Graphics

TL;DR

This paper introduces a virtual system for generating training data in deep learning-based fringe projection profilometry, improving depth estimation accuracy from single images and demonstrating good generalization in real experiments.

Contribution

It presents a virtual FPP system for data generation, a new loss function for better depth estimation, and a comparison of U-Net and pix2pix networks for improved profilometry.

Findings

Virtual FPP data enhances network training.

New loss function improves detail restoration.

Networks trained on virtual data generalize well to real data.

Abstract

Multiple works have applied deep learning to fringe projection profilometry (FPP) in recent years. However, to obtain a large amount of data from actual systems for training is still a tricky problem, and moreover, the network design and optimization still worth exploring. In this paper, we introduce computer graphics to build virtual FPP systems in order to generate the desired datasets conveniently and simply. The way of constructing a virtual FPP system is described in detail firstly, and then some key factors to set the virtual FPP system much close to the reality are analyzed. With the aim of accurately estimating the depth image from only one fringe image, we also design a new loss function to enhance the quality of the overall and detailed information restored. And two representative networks, U-Net and pix2pix, are compared in multiple aspects. The real experiments prove the good accuracy and generalization of the network trained by the data from our virtual systems and the designed loss, implying the potential of our method for applications.