Multi-View Photometric Stereo Revisited

TL;DR

This paper introduces a simplified, neural volume rendering-based multi-view photometric stereo method that effectively handles diverse material types, achieving state-of-the-art 3D reconstruction results.

Contribution

It presents a novel neural volume rendering approach for MVPS that integrates uncertainty modeling and works reliably across different material types, including anisotropic and glossy surfaces.

Findings

Achieves state-of-the-art results on benchmark datasets.

Effectively models diverse material types with neural volume rendering.

Outperforms existing MVPS methods in accuracy and robustness.

Abstract

Multi-view photometric stereo (MVPS) is a preferred method for detailed and precise 3D acquisition of an object from images. Although popular methods for MVPS can provide outstanding results, they are often complex to execute and limited to isotropic material objects. To address such limitations, we present a simple, practical approach to MVPS, which works well for isotropic as well as other object material types such as anisotropic and glossy. The proposed approach in this paper exploits the benefit of uncertainty modeling in a deep neural network for a reliable fusion of photometric stereo (PS) and multi-view stereo (MVS) network predictions. Yet, contrary to the recently proposed state-of-the-art, we introduce neural volume rendering methodology for a trustworthy fusion of MVS and PS measurements. The advantage of introducing neural volume rendering is that it helps in the reliable modeling of objects with diverse material types, where existing MVS methods, PS methods, or both may fail. Furthermore, it allows us to work on neural 3D shape representation, which has recently shown outstanding results for many geometric processing tasks. Our suggested new loss function aims to fits the zero level set of the implicit neural function using the most certain MVS and PS network predictions coupled with weighted neural volume rendering cost. The proposed approach shows state-of-the-art results when tested extensively on several benchmark datasets.