Self-calibrating Deep Photometric Stereo Networks

TL;DR

This paper introduces SDPS-Net, a deep learning approach for uncalibrated photometric stereo that accurately recovers shape and light directions in non-Lambertian scenes with unknown reflectances and lighting, outperforming previous methods.

Contribution

The paper presents a novel two-stage deep learning architecture that effectively handles uncalibrated, non-Lambertian photometric stereo without prior reflectance or lighting assumptions.

Findings

Significantly outperforms previous uncalibrated methods on synthetic datasets.

Effective in real-world scenes with unknown reflectances and lighting.

Reduces learning difficulty through intermediate supervision.

Abstract

This paper proposes an uncalibrated photometric stereo method for non-Lambertian scenes based on deep learning. Unlike previous approaches that heavily rely on assumptions of specific reflectances and light source distributions, our method is able to determine both shape and light directions of a scene with unknown arbitrary reflectances observed under unknown varying light directions. To achieve this goal, we propose a two-stage deep learning architecture, called SDPS-Net, which can effectively take advantage of intermediate supervision, resulting in reduced learning difficulty compared to a single-stage model. Experiments on both synthetic and real datasets show that our proposed approach significantly outperforms previous uncalibrated photometric stereo methods.