MVPSNet: Fast Generalizable Multi-view Photometric Stereo

TL;DR

MVPSNet is a fast, generalizable multi-view photometric stereo method that effectively combines multi-light images to improve 3D reconstruction, especially in textureless regions, while being significantly faster than existing neural network-based methods.

Contribution

We introduce MVPSNet, a novel neural network that combines multi-light images for efficient and accurate 3D reconstruction in MVPS, and a synthetic dataset sMVPS for training.

Findings

Achieves similar results to PS-NeRF but 411 times faster.

Effective in textureless regions where traditional methods fail.

Introduces a new synthetic dataset sMVPS for training and evaluation.

Abstract

We propose a fast and generalizable solution to Multi-view Photometric Stereo (MVPS), called MVPSNet. The key to our approach is a feature extraction network that effectively combines images from the same view captured under multiple lighting conditions to extract geometric features from shading cues for stereo matching. We demonstrate these features, termed `Light Aggregated Feature Maps' (LAFM), are effective for feature matching even in textureless regions, where traditional multi-view stereo methods fail. Our method produces similar reconstruction results to PS-NeRF, a state-of-the-art MVPS method that optimizes a neural network per-scene, while being 411$\times$ faster (105 seconds vs. 12 hours) in inference. Additionally, we introduce a new synthetic dataset for MVPS, sMVPS, which is shown to be effective to train a generalizable MVPS method.