Event-based Photometric Stereo via Rotating Illumination and Per-Pixel Learning

TL;DR

This paper introduces an event-based photometric stereo system using a rotating light source and a per-pixel neural network, enabling accurate surface normal estimation under challenging lighting conditions.

Contribution

It presents a novel event-based approach with a rotating illumination setup and a lightweight neural network that does not require calibration, improving robustness and scalability.

Findings

Achieves 7.12% lower mean angular error than existing methods.

Effective in high dynamic range and ambient illumination conditions.

Robust in regions with sparse events and specularities.

Abstract

Photometric stereo is a technique for estimating surface normals using images captured under varying illumination. However, conventional frame-based photometric stereo methods are limited in real-world applications due to their reliance on controlled lighting, and susceptibility to ambient illumination. To address these limitations, we propose an event-based photometric stereo system that leverages an event camera, which is effective in scenarios with continuously varying scene radiance and high dynamic range conditions. Our setup employs a single light source moving along a predefined circular trajectory, eliminating the need for multiple synchronized light sources and enabling a more compact and scalable design. We further introduce a lightweight per-pixel multi-layer neural network that directly predicts surface normals from event signals generated by intensity changes as the light source rotates, without system calibration. Experimental results on benchmark datasets and real-world data collected with our data acquisition system demonstrate the effectiveness of our method, achieving a 7.12\% reduction in mean angular error compared to existing event-based photometric stereo methods. In addition, our method demonstrates robustness in regions with sparse event activity, strong ambient illumination, and scenes affected by specularities.