EVReflex: Dense Time-to-Impact Prediction for Event-based Obstacle Avoidance

TL;DR

EVReflex introduces a novel method combining event camera and lidar data to accurately predict time-to-impact for obstacle avoidance in dynamic scenes, without prior scene knowledge.

Contribution

It presents a new fusion approach of event and depth data for dense time-to-impact prediction, along with a comprehensive dataset for event-based obstacle avoidance.

Findings

Fusion of event and lidar data improves obstacle avoidance accuracy.

Proposed method predicts time-to-impact without scene prior knowledge.

Extensive dataset supports future research in event-based perception.

Abstract

The broad scope of obstacle avoidance has led to many kinds of computer vision-based approaches. Despite its popularity, it is not a solved problem. Traditional computer vision techniques using cameras and depth sensors often focus on static scenes, or rely on priors about the obstacles. Recent developments in bio-inspired sensors present event cameras as a compelling choice for dynamic scenes. Although these sensors have many advantages over their frame-based counterparts, such as high dynamic range and temporal resolution, event-based perception has largely remained in 2D. This often leads to solutions reliant on heuristics and specific to a particular task. We show that the fusion of events and depth overcomes the failure cases of each individual modality when performing obstacle avoidance. Our proposed approach unifies event camera and lidar streams to estimate metric time-to-impact without prior knowledge of the scene geometry or obstacles. In addition, we release an extensive event-based dataset with six visual streams spanning over 700 scanned scenes.