Temporal Dynamics Enhancer for Directly Trained Spiking Object Detectors

TL;DR

This paper introduces the Temporal Dynamics Enhancer (TDE) to improve spiking neural networks' ability to model temporal information, significantly boosting object detection performance while maintaining low energy consumption.

Contribution

The paper proposes TDE with a spiking encoder and attention gating, along with spike-driven attention to enhance temporal modeling and energy efficiency in SNN-based detectors.

Findings

Achieved 57.7% mAP on PASCAL VOC dataset.

Attained 47.6% mAP on EvDET200K dataset.

SDA reduces attention energy consumption to 0.240 times that of traditional modules.

Abstract

Spiking Neural Networks (SNNs), with their brain-inspired spatiotemporal dynamics and spike-driven computation, have emerged as promising energy-efficient alternatives to Artificial Neural Networks (ANNs). However, existing SNNs typically replicate inputs directly or aggregate them into frames at fixed intervals. Such strategies lead to neurons receiving nearly identical stimuli across time steps, severely limiting the model's expressive power, particularly in complex tasks like object detection. In this work, we propose the Temporal Dynamics Enhancer (TDE) to strengthen SNNs' capacity for temporal information modeling. TDE consists of two modules: a Spiking Encoder (SE) that generates diverse input stimuli across time steps, and an Attention Gating Module (AGM) that guides the SE generation based on inter-temporal dependencies. Moreover, to eliminate the high-energy multiplication operations introduced by the AGM, we propose a Spike-Driven Attention (SDA) to reduce attention-related energy consumption. Extensive experiments demonstrate that TDE can be seamlessly integrated into existing SNN-based detectors and consistently outperforms state-of-the-art methods, achieving mAP50-95 scores of 57.7% on the static PASCAL VOC dataset and 47.6% on the neuromorphic EvDET200K dataset. In terms of energy consumption, the SDA consumes only 0.240 times the energy of conventional attention modules.