SFOD: Spiking Fusion Object Detector

TL;DR

This paper introduces SFOD, an efficient SNN-based object detector for event cameras, featuring a novel fusion module and analysis of decoding strategies, achieving state-of-the-art results in classification and detection tasks.

Contribution

The paper presents the first fusion of multi-scale feature maps in SNNs for event camera object detection and analyzes the impact of decoding strategies and loss functions, advancing SNN performance.

Findings

Achieved 93.7% accuracy on NCAR dataset.

Attained 32.1% mAP on GEN1 detection dataset.

Demonstrated the effectiveness of the Spiking Fusion Module.

Abstract

Event cameras, characterized by high temporal resolution, high dynamic range, low power consumption, and high pixel bandwidth, offer unique capabilities for object detection in specialized contexts. Despite these advantages, the inherent sparsity and asynchrony of event data pose challenges to existing object detection algorithms. Spiking Neural Networks (SNNs), inspired by the way the human brain codes and processes information, offer a potential solution to these difficulties. However, their performance in object detection using event cameras is limited in current implementations. In this paper, we propose the Spiking Fusion Object Detector (SFOD), a simple and efficient approach to SNN-based object detection. Specifically, we design a Spiking Fusion Module, achieving the first-time fusion of feature maps from different scales in SNNs applied to event cameras. Additionally, through integrating our analysis and experiments conducted during the pretraining of the backbone network on the NCAR dataset, we delve deeply into the impact of spiking decoding strategies and loss functions on model performance. Thereby, we establish state-of-the-art classification results based on SNNs, achieving 93.7\% accuracy on the NCAR dataset. Experimental results on the GEN1 detection dataset demonstrate that the SFOD achieves a state-of-the-art mAP of 32.1\%, outperforming existing SNN-based approaches. Our research not only underscores the potential of SNNs in object detection with event cameras but also propels the advancement of SNNs. Code is available at https://github.com/yimeng-fan/SFOD.