High-frequency shape recovery from shading by CNN and domain adaptation

TL;DR

This paper introduces a novel learning-based method that uses shading information from a single RGB-D image to recover high-frequency shapes, employing data augmentation and domain adaptation to overcome training data limitations.

Contribution

It presents a new approach combining shape from shading with deep learning and domain adaptation to recover detailed high-frequency shapes from single images.

Findings

Effective high-frequency shape recovery demonstrated

Data augmentation improves model performance

Method outperforms traditional shape from shading techniques

Abstract

Importance of structured-light based one-shot scanning technique is increasing because of its simple system configuration and ability of capturing moving objects. One severe limitation of the technique is that it can capture only sparse shape, but not high frequency shapes, because certain area of projection pattern is required to encode spatial information. In this paper, we propose a technique to recover high-frequency shapes by using shading information, which is captured by one-shot RGB-D sensor based on structured light with single camera. Since color image comprises shading information of object surface, high-frequency shapes can be recovered by shape from shading techniques. Although multiple images with different lighting positions are required for shape from shading techniques, we propose a learning based approach to recover shape from a single image. In addition, to overcome the problem of preparing sufficient amount of data for training, we propose a new data augmentation method for high-frequency shapes using synthetic data and domain adaptation. Experimental results are shown to confirm the effectiveness of the proposed method.