Event-based Motion-Robust Accurate Shape Estimation for Mixed Reflectance Scenes

TL;DR

This paper introduces a fast, motion-robust event-based 3D shape estimation system capable of accurately capturing mixed reflectance scenes by decomposing reflections and leveraging diffuse parts as a virtual screen.

Contribution

The work presents a novel event-based structured light system that decomposes mixed reflections and achieves high-accuracy, fast 3D imaging of complex scenes using epipolar constraints and deflectometry.

Findings

Achieves < 600 μm depth error in mixed scenes.

Operates at 14Hz with high motion robustness.

Offers an ultrafast 250Hz capture mode for diffuse scenes.

Abstract

Event-based structured light systems have recently been introduced as an exciting alternative to conventional frame-based triangulation systems for the 3D measurements of diffuse surfaces. Important benefits include the fast capture speed and the high dynamic range provided by the event camera - albeit at the cost of lower data quality. So far, both low-accuracy event-based and high-accuracy frame-based 3D imaging systems are tailored to a specific surface type, such as diffuse or specular, and can not be used for a broader class of object surfaces ("mixed reflectance scenes"). In this work, we present a novel event-based structured light system that enables fast 3D imaging of mixed reflectance scenes with high accuracy. On the captured events, we use epipolar constraints that intrinsically enable decomposing the measured reflections into diffuse, two-bounce specular, and other multi-bounce reflections. The diffuse surfaces in the scene are reconstructed using triangulation. Then, the reconstructed diffuse scene parts are leveraged as a "display" to evaluate the specular scene parts via deflectometry. This novel procedure allows us to use the entire scene as a virtual screen, using only a scanning laser and an event camera. The resulting system achieves fast and motion-robust (14Hz) reconstructions of mixed reflectance scenes with < 600 ${\mu}m$ depth error. Moreover, we introduce an "ultrafast" capture mode (250Hz) for the 3D measurement of diffuse scenes.