MSF-Net: Multi-Stage Feature Extraction and Fusion for Robust Photometric Stereo

TL;DR

MSF-Net introduces a multi-stage feature extraction and fusion framework for photometric stereo, significantly improving surface normal estimation accuracy by capturing detailed features and enhancing feature interaction.

Contribution

The paper presents MSF-Net, a novel multi-stage feature extraction and fusion framework with a selective update strategy for improved photometric stereo performance.

Findings

Outperforms previous methods on DiLiGenT benchmark

Achieves higher accuracy in surface normal estimation

Effectively captures intricate surface details

Abstract

Photometric stereo is a technique aimed at determining surface normals through the utilization of shading cues derived from images taken under different lighting conditions. However, existing learning-based approaches often fail to accurately capture features at multiple stages and do not adequately promote interaction between these features. Consequently, these models tend to extract redundant features, especially in areas with intricate details such as wrinkles and edges. To tackle these issues, we propose MSF-Net, a novel framework for extracting information at multiple stages, paired with selective update strategy, aiming to extract high-quality feature information, which is critical for accurate normal construction. Additionally, we have developed a feature fusion module to improve the interplay among different features. Experimental results on the DiLiGenT benchmark show that our proposed MSF-Net significantly surpasses previous state-of-the-art methods in the accuracy of surface normal estimation.