DeepShadow: Neural Shape from Shadow

TL;DR

DeepShadow introduces a neural network approach that uniquely leverages shadows in photometric stereo images to accurately recover 3D shape and surface normals without pre-training, outperforming traditional methods in some cases.

Contribution

It is the first method to use neural networks for 3D shape-from-shadows, utilizing shadows as a strong learning signal without needing pre-training or labeled data.

Findings

Reconstructs 3D shape from shadows alone.

Outperforms shape-from-shading in certain scenarios.

Does not require pre-training or labeled datasets.

Abstract

This paper presents DeepShadow, a one-shot method for recovering the depth map and surface normals from photometric stereo shadow maps. Previous works that try to recover the surface normals from photometric stereo images treat cast shadows as a disturbance. We show that the self and cast shadows not only do not disturb 3D reconstruction, but can be used alone, as a strong learning signal, to recover the depth map and surface normals. We demonstrate that 3D reconstruction from shadows can even outperform shape-from-shading in certain cases. To the best of our knowledge, our method is the first to reconstruct 3D shape-from-shadows using neural networks. The method does not require any pre-training or expensive labeled data, and is optimized during inference time.