PS-FCN: A Flexible Learning Framework for Photometric Stereo

TL;DR

This paper introduces PS-FCN, a deep learning framework for photometric stereo that handles arbitrary light directions and multiple images without requiring predefined light sets, demonstrating superior performance on real datasets.

Contribution

We propose PS-FCN, a flexible convolutional network that generalizes well to real-world data and uncalibrated scenarios without needing predefined light directions.

Findings

Outperforms existing methods on real datasets in calibrated photometric stereo.

Successfully extends to uncalibrated photometric stereo.

Handles arbitrary number of images and light directions in an order-agnostic way.

Abstract

This paper addresses the problem of photometric stereo for non-Lambertian surfaces. Existing approaches often adopt simplified reflectance models to make the problem more tractable, but this greatly hinders their applications on real-world objects. In this paper, we propose a deep fully convolutional network, called PS-FCN, that takes an arbitrary number of images of a static object captured under different light directions with a fixed camera as input, and predicts a normal map of the object in a fast feed-forward pass. Unlike the recently proposed learning based method, PS-FCN does not require a pre-defined set of light directions during training and testing, and can handle multiple images and light directions in an order-agnostic manner. Although we train PS-FCN on synthetic data, it can generalize well on real datasets. We further show that PS-FCN can be easily extended to handle the problem of uncalibrated photometric stereo.Extensive experiments on public real datasets show that PS-FCN outperforms existing approaches in calibrated photometric stereo, and promising results are achieved in uncalibrated scenario, clearly demonstrating its effectiveness.