PEOD: A Pixel-Aligned Event-RGB Benchmark for Object Detection under Challenging Conditions

TL;DR

PEOD introduces a large-scale, high-resolution Event-RGB dataset designed to evaluate object detection in challenging conditions, enabling comprehensive benchmarking of multimodal perception methods.

Contribution

This paper presents PEOD, the first high-resolution, pixel-aligned Event-RGB dataset with extensive challenging scenario coverage, and provides benchmark results for 14 detection methods.

Findings

Fusion models perform best on normal conditions.

Event-based models excel under severe illumination challenges.

Existing fusion methods have limitations with degraded frame quality.

Abstract

Robust object detection for challenging scenarios increasingly relies on event cameras, yet existing Event-RGB datasets remain constrained by sparse coverage of extreme conditions and low spatial resolution (<= 640 x 480), which prevents comprehensive evaluation of detectors under challenging scenarios. To address these limitations, we propose PEOD, the first large-scale, pixel-aligned and high-resolution (1280 x 720) Event-RGB dataset for object detection under challenge conditions. PEOD contains 130+ spatiotemporal-aligned sequences and 340k manual bounding boxes, with 57% of data captured under low-light, overexposure, and high-speed motion. Furthermore, we benchmark 14 methods across three input configurations (Event-based, RGB-based, and Event-RGB fusion) on PEOD. On the full test set and normal subset, fusion-based models achieve the excellent performance. However, in illumination challenge subset, the top event-based model outperforms all fusion models, while fusion models still outperform their RGB-based counterparts, indicating limits of existing fusion methods when the frame modality is severely degraded. PEOD establishes a realistic, high-quality benchmark for multimodal perception and facilitates future research.