NPLMV-PS: Neural Point-Light Multi-View Photometric Stereo

TL;DR

This paper introduces NPLMV-PS, a neural multi-view photometric stereo method that explicitly models per-pixel radiance and shadows, achieving high accuracy and robustness in 3D reconstruction compared to classical and neural approaches.

Contribution

It presents a novel neural MVPS approach that leverages explicit radiance modeling and shadow ray tracing, outperforming existing methods on benchmark datasets.

Findings

Achieves 0.2mm Chamfer distance on DiLiGenT-MV benchmark.

Outperforms classical MVPS methods and SOTA neural approaches.

Shows robustness with sparse view and lighting setups.

Abstract

In this work we present a novel multi-view photometric stereo (MVPS) method. Like many works in 3D reconstruction we are leveraging neural shape representations and learnt renderers. However, our work differs from the state-of-the-art multi-view PS methods such as PS-NeRF or Supernormal in that we explicitly leverage per-pixel intensity renderings rather than relying mainly on estimated normals. We model point light attenuation and explicitly raytrace cast shadows in order to best approximate the incoming radiance for each point. The estimated incoming radiance is used as input to a fully neural material renderer that uses minimal prior assumptions and it is jointly optimised with the surface. Estimated normals and segmentation maps are also incorporated in order to maximise the surface accuracy. Our method is among the first (along with Supernormal) to outperform the classical MVPS approach proposed by the DiLiGenT-MV benchmark and achieves average 0.2mm Chamfer distance for objects imaged at approx 1.5m distance away with approximate 400x400 resolution. Moreover, our method shows high robustness to the sparse MVPS setup (6 views, 6 lights) greatly outperforming the SOTA competitor (0.38mm vs 0.61mm), illustrating the importance of neural rendering in multi-view photometric stereo.