LIPIDS: Learning-based Illumination Planning In Discretized (Light) Space for Photometric Stereo

TL;DR

LIPIDS introduces a learning-based method to optimize lighting configurations in photometric stereo, significantly improving normal estimation accuracy by selecting optimal light directions from discretized space.

Contribution

The paper presents LIPIDS, a novel learning-based framework that optimally plans illumination in discretized light space for enhanced photometric stereo performance.

Findings

LIPIDS outperforms existing illumination planning methods on synthetic datasets.

LIPIDS achieves comparable results to state-of-the-art methods on real-world datasets.

The learned lighting configurations improve normal estimation accuracy.

Abstract

Photometric stereo is a powerful method for obtaining per-pixel surface normals from differently illuminated images of an object. While several methods address photometric stereo with different image (or light) counts ranging from one to two to a hundred, very few focus on learning optimal lighting configuration. Finding an optimal configuration is challenging due to the vast number of possible lighting directions. Moreover, exhaustively sampling all possibilities is impractical due to time and resource constraints. Photometric stereo methods have demonstrated promising performance on existing datasets, which feature limited light directions sparsely sampled from the light space. Therefore, can we optimally utilize these datasets for illumination planning? In this work, we introduce LIPIDS - Learning-based Illumination Planning In Discretized light Space to achieve minimal and optimal lighting configurations for photometric stereo under arbitrary light distribution. We propose a Light Sampling Network (LSNet) that optimizes lighting direction for a fixed number of lights by minimizing the normal loss through a normal regression network. The learned light configurations can directly estimate surface normals during inference, even using an off-the-shelf photometric stereo method. Extensive qualitative and quantitative analyses on synthetic and real-world datasets show that photometric stereo under learned lighting configurations through LIPIDS either surpasses or is nearly comparable to existing illumination planning methods across different photometric stereo backbones.