Physics-Free Spectrally Multiplexed Photometric Stereo under Unknown Spectral Composition

TL;DR

This paper introduces a novel physics-free neural network approach, SpectraM-PS, for spectrally multiplexed photometric stereo that recovers surface normals without calibration or spectral knowledge, supported by a new benchmark dataset.

Contribution

It presents the first physics-free network architecture for uncalibrated spectrally multiplexed photometric stereo and introduces the SpectraM14 dataset for evaluation.

Findings

Effective surface normal recovery across diverse conditions.

No need for calibrated lighting or spectral information.

Benchmark dataset enables comprehensive evaluation.

Abstract

In this paper, we present a groundbreaking spectrally multiplexed photometric stereo approach for recovering surface normals of dynamic surfaces without the need for calibrated lighting or sensors, a notable advancement in the field traditionally hindered by stringent prerequisites and spectral ambiguity. By embracing spectral ambiguity as an advantage, our technique enables the generation of training data without specialized multispectral rendering frameworks. We introduce a unique, physics-free network architecture, SpectraM-PS, that effectively processes multiplexed images to determine surface normals across a wide range of conditions and material types, without relying on specific physically-based knowledge. Additionally, we establish the first benchmark dataset, SpectraM14, for spectrally multiplexed photometric stereo, facilitating comprehensive evaluations against existing calibrated methods. Our contributions significantly enhance the capabilities for dynamic surface recovery, particularly in uncalibrated setups, marking a pivotal step forward in the application of photometric stereo across various domains.