TS-SNN: Temporal Shift Module for Spiking Neural Networks

TL;DR

This paper introduces the Temporal Shift module for Spiking Neural Networks (TS-SNN), enhancing their ability to utilize temporal information efficiently, achieving state-of-the-art results with minimal additional computational cost.

Contribution

The paper proposes a novel Temporal Shift module for SNNs that integrates temporal features with minimal overhead, improving performance and energy efficiency.

Findings

Achieves 96.72% on CIFAR-10

Outperforms previous SNN models on ImageNet

Requires only one additional learnable parameter

Abstract

Spiking Neural Networks (SNNs) are increasingly recognized for their biological plausibility and energy efficiency, positioning them as strong alternatives to Artificial Neural Networks (ANNs) in neuromorphic computing applications. SNNs inherently process temporal information by leveraging the precise timing of spikes, but balancing temporal feature utilization with low energy consumption remains a challenge. In this work, we introduce Temporal Shift module for Spiking Neural Networks (TS-SNN), which incorporates a novel Temporal Shift (TS) module to integrate past, present, and future spike features within a single timestep via a simple yet effective shift operation. A residual combination method prevents information loss by integrating shifted and original features. The TS module is lightweight, requiring only one additional learnable parameter, and can be seamlessly integrated into existing architectures with minimal additional computational cost. TS-SNN achieves state-of-the-art performance on benchmarks like CIFAR-10 (96.72\%), CIFAR-100 (80.28\%), and ImageNet (70.61\%) with fewer timesteps, while maintaining low energy consumption. This work marks a significant step forward in developing efficient and accurate SNN architectures.