Ego-motion Estimation Based on Fusion of Images and Events

TL;DR

This paper introduces EAS, a novel data fusion algorithm combining images and event streams from bio-inspired sensors, significantly improving ego-motion estimation accuracy in challenging lighting conditions.

Contribution

The paper presents a new fusion algorithm, EAS, that effectively combines images and event data to enhance ego-motion estimation, especially in dim scenes.

Findings

EAS achieves the highest accuracy among event representations.

EAS reduces average APE by 69% compared to original images.

The algorithm leverages high dynamic range of event cameras for better performance.

Abstract

Event camera is a novel bio-inspired vision sensor that outputs event stream. In this paper, we propose a novel data fusion algorithm called EAS to fuse conventional intensity images with the event stream. The fusion result is applied to some ego-motion estimation frameworks, and is evaluated on a public dataset acquired in dim scenes. In our 3-DoF rotation estimation framework, EAS achieves the highest estimation accuracy among intensity images and representations of events including event slice, TS and SITS. Compared with original images, EAS reduces the average APE by 69%, benefiting from the inclusion of more features for tracking. The result shows that our algorithm effectively leverages the high dynamic range of event cameras to improve the performance of the ego-motion estimation framework based on optical flow tracking in difficult illumination conditions.