Leveraging Spatial and Photometric Context for Calibrated Non-Lambertian Photometric Stereo

TL;DR

This paper introduces a novel fully-convolutional neural network architecture that simultaneously leverages spatial and photometric context for improved surface shape estimation in non-Lambertian photometric stereo, achieving better efficiency and accuracy.

Contribution

It proposes a separable 4D convolutional architecture that combines spatial and photometric information, reducing model size and improving inference efficiency over existing methods.

Findings

Outperforms existing methods in accuracy on real-world benchmarks.

Achieves higher efficiency with smaller network size.

Effectively captures both spatial and photometric context for shape estimation.

Abstract

The problem of estimating a surface shape from its observed reflectance properties still remains a challenging task in computer vision. The presence of global illumination effects such as inter-reflections or cast shadows makes the task particularly difficult for non-convex real-world surfaces. State-of-the-art methods for calibrated photometric stereo address these issues using convolutional neural networks (CNNs) that primarily aim to capture either the spatial context among adjacent pixels or the photometric one formed by illuminating a sample from adjacent directions. In this paper, we bridge these two objectives and introduce an efficient fully-convolutional architecture that can leverage both spatial and photometric context simultaneously. In contrast to existing approaches that rely on standard 2D CNNs and regress directly to surface normals, we argue that using separable 4D convolutions and regressing to 2D Gaussian heat-maps severely reduces the size of the network and makes inference more efficient. Our experimental results on a real-world photometric stereo benchmark show that the proposed approach outperforms the existing methods both in efficiency and accuracy.