RotatedMVPS: Multi-view Photometric Stereo with Rotated Natural Light

TL;DR

RotatedMVPS introduces a practical multi-view photometric stereo method that recovers high-fidelity shapes and reflectances under natural, rotated lighting conditions by leveraging light consistency and learning-based priors.

Contribution

It is the first MVPS method to handle rotated natural illumination with a practical setup and integrates learning-based priors to improve accuracy.

Findings

Effective shape and reflectance recovery on synthetic datasets.

Robust performance on real-world datasets.

Outperforms existing MVPS methods in natural lighting scenarios.

Abstract

Multiview photometric stereo (MVPS) seeks to recover high-fidelity surface shapes and reflectances from images captured under varying views and illuminations. However, existing MVPS methods often require controlled darkroom settings for varying illuminations or overlook the recovery of reflectances and illuminations properties, limiting their applicability in natural illumination scenarios and downstream inverse rendering tasks. In this paper, we propose RotatedMVPS to solve shape and reflectance recovery under rotated natural light, achievable with a practical rotation stage. By ensuring light consistency across different camera and object poses, our method reduces the unknowns associated with complex environment light. Furthermore, we integrate data priors from off-the-shelf learning-based single-view photometric stereo methods into our MVPS framework, significantly enhancing the accuracy of shape and reflectance recovery. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness of our approach.