Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset

TL;DR

This paper presents a novel MoE heat conduction-based object detection algorithm for event streams, achieving a balance between accuracy and efficiency, and introduces a new large-scale event-based dataset for benchmarking.

Contribution

The paper introduces a new MoE heat conduction-based detection method and a comprehensive event-based dataset, advancing the state-of-the-art in event-based object detection.

Findings

The proposed MoE-HCO detector outperforms existing methods in accuracy and efficiency.

EvDET200K dataset provides extensive data for training and benchmarking.

Over 15 state-of-the-art detectors evaluated on the new dataset.

Abstract

Object detection in event streams has emerged as a cutting-edge research area, demonstrating superior performance in low-light conditions, scenarios with motion blur, and rapid movements. Current detectors leverage spiking neural networks, Transformers, or convolutional neural networks as their core architectures, each with its own set of limitations including restricted performance, high computational overhead, or limited local receptive fields. This paper introduces a novel MoE (Mixture of Experts) heat conduction-based object detection algorithm that strikingly balances accuracy and computational efficiency. Initially, we employ a stem network for event data embedding, followed by processing through our innovative MoE-HCO blocks. Each block integrates various expert modules to mimic heat conduction within event streams. Subsequently, an IoU-based query selection module is utilized for efficient token extraction, which is then channeled into a detection head for the final object detection process. Furthermore, we are pleased to introduce EvDET200K, a novel benchmark dataset for event-based object detection. Captured with a high-definition Prophesee EVK4-HD event camera, this dataset encompasses 10 distinct categories, 200,000 bounding boxes, and 10,054 samples, each spanning 2 to 5 seconds. We also provide comprehensive results from over 15 state-of-the-art detectors, offering a solid foundation for future research and comparison. The source code of this paper will be released on: https://github.com/Event-AHU/OpenEvDET