Standard and Event Cameras Fusion for Dense Mapping

TL;DR

This paper introduces a fusion method combining event and standard cameras to produce dense 3D maps, leveraging the high dynamic range of event sensors and the completeness of standard frames.

Contribution

A novel fusion strategy that generates dense depth maps by combining event-based edge detection with frame-based filling, improving mapping density.

Findings

Increased density of 3D maps compared to semi-dense methods

Effective use of filling score to evaluate map quality

Enhanced mapping performance through sensor fusion

Abstract

Event cameras are a kind of bio-inspired sensors that generate data when the brightness changes, which are of low-latency and high dynamic range (HDR). However, due to the nature of the sparse event stream, event-based mapping can only obtain sparse or semi-dense edge 3D maps. By contrast, standard cameras provide complete frames. To leverage the complementarity of event-based and standard frame-based cameras, we propose a fusion strategy for dense mapping in this paper. We first generate an edge map from events, and then fill the map using frames to obtain the dense depth map. We propose "filling score" to evaluate the quality of filled results and show that our strategy can increase the number of existing semi-dense 3D map.