Scalable, Detailed and Mask-Free Universal Photometric Stereo

TL;DR

This paper presents SDM-UniPS, a scalable and mask-free universal photometric stereo network capable of recovering detailed surface normals under unknown, complex lighting conditions, outperforming existing methods with fewer images.

Contribution

The paper introduces a novel photometric stereo network that extracts spatial-light features and models non-local interactions, along with a new synthetic dataset for training.

Findings

Surpasses calibrated lighting-specific methods on benchmarks.

Achieves high-quality normal maps with fewer images.

Operates effectively without object masks.

Abstract

In this paper, we introduce SDM-UniPS, a groundbreaking Scalable, Detailed, Mask-free, and Universal Photometric Stereo network. Our approach can recover astonishingly intricate surface normal maps, rivaling the quality of 3D scanners, even when images are captured under unknown, spatially-varying lighting conditions in uncontrolled environments. We have extended previous universal photometric stereo networks to extract spatial-light features, utilizing all available information in high-resolution input images and accounting for non-local interactions among surface points. Moreover, we present a new synthetic training dataset that encompasses a diverse range of shapes, materials, and illumination scenarios found in real-world scenes. Through extensive evaluation, we demonstrate that our method not only surpasses calibrated, lighting-specific techniques on public benchmarks, but also excels with a significantly smaller number of input images even without object masks.