NeuralMPS: Non-Lambertian Multispectral Photometric Stereo via Spectral Reflectance Decomposition

TL;DR

NeuralMPS introduces a neural network approach for multispectral photometric stereo that handles non-Lambertian surfaces by spectral reflectance decomposition, enabling accurate surface normal recovery in complex real-world scenarios.

Contribution

The paper proposes a spectral reflectance decomposition model and a neural network that extends photometric stereo to non-Lambertian multispectral surfaces, overcoming previous Lambertian limitations.

Findings

Effective in synthetic and real-world scenes

Outperforms Lambertian-based methods

Handles complex spectral reflectance

Abstract

Multispectral photometric stereo(MPS) aims at recovering the surface normal of a scene from a single-shot multispectral image captured under multispectral illuminations. Existing MPS methods adopt the Lambertian reflectance model to make the problem tractable, but it greatly limits their application to real-world surfaces. In this paper, we propose a deep neural network named NeuralMPS to solve the MPS problem under general non-Lambertian spectral reflectances. Specifically, we present a spectral reflectance decomposition(SRD) model to disentangle the spectral reflectance into geometric components and spectral components. With this decomposition, we show that the MPS problem for surfaces with a uniform material is equivalent to the conventional photometric stereo(CPS) with unknown light intensities. In this way, NeuralMPS reduces the difficulty of the non-Lambertian MPS problem by leveraging the well-studied non-Lambertian CPS methods. Experiments on both synthetic and real-world scenes demonstrate the effectiveness of our method.