Event-based Shape from Polarization

TL;DR

This paper introduces an event-based method for Shape-from-Polarization that leverages high-speed event cameras to improve accuracy and speed, outperforming traditional frame-based approaches in synthetic and real-world scenarios.

Contribution

It proposes a novel event-based setup with a rotating polarizer and a learning-based approach to estimate surface normals at high speeds and low event-rates, surpassing existing physics-based methods.

Findings

Reduces MAE by 25% compared to frame-based baselines

Improves surface normal estimation accuracy by 52% with learning-based approach

Achieves 50 fps acquisition speed, doubling the framerate of commercial sensors

Abstract

State-of-the-art solutions for Shape-from-Polarization (SfP) suffer from a speed-resolution tradeoff: they either sacrifice the number of polarization angles measured or necessitate lengthy acquisition times due to framerate constraints, thus compromising either accuracy or latency. We tackle this tradeoff using event cameras. Event cameras operate at microseconds resolution with negligible motion blur, and output a continuous stream of events that precisely measures how light changes over time asynchronously. We propose a setup that consists of a linear polarizer rotating at high-speeds in front of an event camera. Our method uses the continuous event stream caused by the rotation to reconstruct relative intensities at multiple polarizer angles. Experiments demonstrate that our method outperforms physics-based baselines using frames, reducing the MAE by 25% in synthetic and real-world dataset. In the real world, we observe, however, that the challenging conditions (i.e., when few events are generated) harm the performance of physics-based solutions. To overcome this, we propose a learning-based approach that learns to estimate surface normals even at low event-rates, improving the physics-based approach by 52% on the real world dataset. The proposed system achieves an acquisition speed equivalent to 50 fps (>twice the framerate of the commercial polarization sensor) while retaining the spatial resolution of 1MP. Our evaluation is based on the first large-scale dataset for event-based SfP