MS-PS: A Multi-Scale Network for Photometric Stereo With a New Comprehensive Training Dataset

TL;DR

This paper introduces a multi-scale neural network architecture for photometric stereo, supported by a large, synthetic dataset with challenging materials, achieving state-of-the-art 3D surface reconstruction accuracy.

Contribution

The paper presents a novel multi-scale network architecture and a comprehensive synthetic dataset for photometric stereo, enhancing reconstruction accuracy and flexibility.

Findings

Achieved state-of-the-art normal estimation accuracy.

The dataset includes complex materials like metals and glass.

The method is robust to varying image numbers and sizes.

Abstract

The photometric stereo (PS) problem consists in reconstructing the 3D-surface of an object, thanks to a set of photographs taken under different lighting directions. In this paper, we propose a multi-scale architecture for PS which, combined with a new dataset, yields state-of-the-art results. Our proposed architecture is flexible: it permits to consider a variable number of images as well as variable image size without loss of performance. In addition, we define a set of constraints to allow the generation of a relevant synthetic dataset to train convolutional neural networks for the PS problem. Our proposed dataset is much larger than pre-existing ones, and contains many objects with challenging materials having anisotropic reflectance (e.g. metals, glass). We show on publicly available benchmarks that the combination of both these contributions drastically improves the accuracy of the estimated normal field, in comparison with previous state-of-the-art methods.