Semi-Dense 3D Reconstruction with a Stereo Event Camera

TL;DR

This paper introduces a novel stereo event-camera based 3D reconstruction method that optimizes an energy function and employs probabilistic depth fusion, outperforming existing approaches in various scene types.

Contribution

It presents a new approach combining energy optimization and probabilistic depth fusion for dense 3D reconstruction from stereo event cameras without prior scene or motion constraints.

Findings

Outperforms state-of-the-art stereo event-based methods

Effective in both textured and sparse scenes

No special motion or scene prior required

Abstract

Event cameras are bio-inspired sensors that offer several advantages, such as low latency, high-speed and high dynamic range, to tackle challenging scenarios in computer vision. This paper presents a solution to the problem of 3D reconstruction from data captured by a stereo event-camera rig moving in a static scene, such as in the context of stereo Simultaneous Localization and Mapping. The proposed method consists of the optimization of an energy function designed to exploit small-baseline spatio-temporal consistency of events triggered across both stereo image planes. To improve the density of the reconstruction and to reduce the uncertainty of the estimation, a probabilistic depth-fusion strategy is also developed. The resulting method has no special requirements on either the motion of the stereo event-camera rig or on prior knowledge about the scene. Experiments demonstrate our method can deal with both texture-rich scenes as well as sparse scenes, outperforming state-of-the-art stereo methods based on event data image representations.