Event-Based Dense Reconstruction Pipeline

TL;DR

This paper presents a pipeline that leverages event cameras and deep learning to achieve dense 3D scene reconstruction, overcoming the semi-dense limitations of traditional event-based methods.

Contribution

It introduces a novel event-based dense reconstruction pipeline combining deep learning, SfM, and MVS techniques for the first time.

Findings

Successfully reconstructs dense 3D maps from event data

Outperforms semi-dense methods in detail and completeness

Integrates deep learning with traditional 3D reconstruction pipelines

Abstract

Event cameras are a new type of sensors that are different from traditional cameras. Each pixel is triggered asynchronously by event. The trigger event is the change of the brightness irradiated on the pixel. If the increment or decrement of brightness is higher than a certain threshold, an event is output. Compared with traditional cameras, event cameras have the advantages of high dynamic range and no motion blur. Since events are caused by the apparent motion of intensity edges, the majority of 3D reconstructed maps consist only of scene edges, i.e., semi-dense maps, which is not enough for some applications. In this paper, we propose a pipeline to realize event-based dense reconstruction. First, deep learning is used to reconstruct intensity images from events. And then, structure from motion (SfM) is used to estimate camera intrinsic, extrinsic and sparse point cloud. Finally, multi-view stereo (MVS) is used to complete dense reconstruction.