Real-time event simulation with frame-based cameras

TL;DR

This paper introduces a novel event camera simulator capable of generating event streams in real-time, significantly enhancing the practicality of using simulated data for time-sensitive robotics and computer vision applications.

Contribution

The work presents a new simulation method that achieves two orders of magnitude faster performance while maintaining high-quality event stream generation.

Findings

Simulation speed improved by 100x for real-time applications.

Maintains competitive quality of event streams compared to offline simulators.

Enables real-time testing and development without physical event cameras.

Abstract

Event cameras are becoming increasingly popular in robotics and computer vision due to their beneficial properties, e.g., high temporal resolution, high bandwidth, almost no motion blur, and low power consumption. However, these cameras remain expensive and scarce in the market, making them inaccessible to the majority. Using event simulators minimizes the need for real event cameras to develop novel algorithms. However, due to the computational complexity of the simulation, the event streams of existing simulators cannot be generated in real-time but rather have to be pre-calculated from existing video sequences or pre-rendered and then simulated from a virtual 3D scene. Although these offline generated event streams can be used as training data for learning tasks, all response time dependent applications cannot benefit from these simulators yet, as they still require an actual event camera. This work proposes simulation methods that improve the performance of event simulation by two orders of magnitude (making them real-time capable) while remaining competitive in the quality assessment.