Deep Uncalibrated Photometric Stereo via Inter-Intra Image Feature Fusion

TL;DR

This paper introduces a novel deep learning approach for uncalibrated photometric stereo that leverages inter-intra image feature fusion to improve surface normal estimation, especially on dark materials, outperforming existing methods.

Contribution

The paper proposes an inter-intra image feature fusion module that effectively utilizes multi-image correlations to enhance normal estimation in uncalibrated photometric stereo.

Findings

Significantly better results than state-of-the-art on synthetic data.

Improved normal estimation on dark materials.

Effective utilization of inter-image information enhances performance.

Abstract

Uncalibrated photometric stereo is proposed to estimate the detailed surface normal from images under varying and unknown lightings. Recently, deep learning brings powerful data priors to this underdetermined problem. This paper presents a new method for deep uncalibrated photometric stereo, which efficiently utilizes the inter-image representation to guide the normal estimation. Previous methods use optimization-based neural inverse rendering or a single size-independent pooling layer to deal with multiple inputs, which are inefficient for utilizing information among input images. Given multi-images under different lighting, we consider the intra-image and inter-image variations highly correlated. Motivated by the correlated variations, we designed an inter-intra image feature fusion module to introduce the inter-image representation into the per-image feature extraction. The extra representation is used to guide the per-image feature extraction and eliminate the ambiguity in normal estimation. We demonstrate the effect of our design on a wide range of samples, especially on dark materials. Our method produces significantly better results than the state-of-the-art methods on both synthetic and real data.