TSkips: Efficiency Through Explicit Temporal Delay Connections in Spiking Neural Networks

TL;DR

TSkips introduces explicit temporal delay connections in SNNs, combined with neural architecture search, to improve performance on sequential event-based tasks by capturing long-term spatio-temporal dependencies.

Contribution

The paper proposes TSkips, a novel SNN architecture with delay-augmented skip connections optimized via NAS, enhancing temporal dependency modeling in event-based data processing.

Findings

Up to 18% reduction in AEE on DSEC-flow

8% increase in accuracy on DVS128 Gesture

Up to 16% higher accuracy on SSC

Abstract

Spiking Neural Networks (SNNs) with their bio-inspired Leaky Integrate-and-Fire (LIF) neurons inherently capture temporal information. This makes them well-suited for sequential tasks like processing event-based data from Dynamic Vision Sensors (DVS) and event-based speech tasks. Harnessing the temporal capabilities of SNNs requires mitigating vanishing spikes during training, capturing spatio-temporal patterns and enhancing precise spike timing. To address these challenges, we propose TSkips, augmenting SNN architectures with forward and backward skip connections that incorporate explicit temporal delays. These connections capture long-term spatio-temporal dependencies and facilitate better spike flow over long sequences. The introduction of TSkips creates a vast search space of possible configurations, encompassing skip positions and time delay values. To efficiently navigate this search space, this work leverages training-free Neural Architecture Search (NAS) to identify optimal network structures and corresponding delays. We demonstrate the effectiveness of our approach on four event-based datasets: DSEC-flow for optical flow estimation, DVS128 Gesture for hand gesture recognition and Spiking Heidelberg Digits (SHD) and Spiking Speech Commands (SSC) for speech recognition. Our method achieves significant improvements across these datasets: up to 18% reduction in Average Endpoint Error (AEE) on DSEC-flow, 8% increase in classification accuracy on DVS128 Gesture, and up to 8% and 16% higher classification accuracy on SHD and SSC, respectively.