# Efficient and real-time perception: a survey on end-to-end event-based object detection in autonomous driving

## TL;DR

This survey explores end-to-end object detection using event-based cameras for autonomous driving, highlighting challenges and recent advancements in processing this novel data format.

## Contribution

The paper provides a comprehensive survey of end-to-end event-based object detection methods, hardware, and datasets for autonomous driving.

## Key findings

- Event-based cameras offer advantages like high dynamic range and low power consumption but require specialized algorithms.
- Current models adapted from frame-based systems often fail to fully utilize event data's unique properties.
- The survey evaluates system-level throughput using RTX 4090 GPU for several state-of-the-art models on GEN1 and 1MP datasets.

## Abstract

Autonomous driving has the potential to enhance driving comfort and accessibility, reduce accidents, and improve road safety, with vision sensors playing a key role in enabling vehicle autonomy. Among existing sensors, event-based cameras offer advantages such as a high dynamic range, low power consumption, and enhanced motion detection capabilities compared to traditional frame-based cameras. However, their sparse and asynchronous data present unique processing challenges that require specialized algorithms and hardware. While some models originally developed for frame-based inputs have been adapted to handle event data, they often fail to fully exploit the distinct properties of this novel data format, primarily due to its fundamental structural differences. As a result, new algorithms, including neuromorphic, have been developed specifically for event data. Many of these models are still in the early stages and often lack the maturity and accuracy of traditional approaches. This survey paper focuses on end-to-end event-based object detection for autonomous driving, covering key aspects such as sensing and processing hardware designs, datasets, and algorithms, including dense, spiking, and graph-based neural networks, along with relevant encoding and pre-processing techniques. In addition, this work highlights the shortcomings in the evaluation practices to ensure fair and meaningful comparisons across different event data processing approaches and hardware platforms. Within the scope of this survey, system-level throughput was evaluated from raw event data to model output on an RTX 4090 24GB GPU for several state-of-the-art models using the GEN1 and 1MP datasets. The study also includes a discussion and outlines potential directions for future research.