Advancing Spiking Neural Networks towards Multiscale Spatiotemporal Interaction Learning

TL;DR

This paper introduces a multiscale spatiotemporal interaction learning framework for Spiking Neural Networks, significantly improving their performance and bridging the gap with traditional ANNs by capturing complex event data correlations.

Contribution

The paper proposes a novel Spiking Multiscale Attention module and Attention ZoneOut regularization to enhance SNNs' ability to learn multiscale spatiotemporal information.

Findings

Achieved state-of-the-art results on neural morphology datasets.

Reached 77.1% accuracy on ImageNet-1K with a 104-layer ResNet.

Demonstrated the effectiveness of multiscale attention in SNNs.

Abstract

Recent advancements in neuroscience research have propelled the development of Spiking Neural Networks (SNNs), which not only have the potential to further advance neuroscience research but also serve as an energy-efficient alternative to Artificial Neural Networks (ANNs) due to their spike-driven characteristics. However, previous studies often neglected the multiscale information and its spatiotemporal correlation between event data, leading SNN models to approximate each frame of input events as static images. We hypothesize that this oversimplification significantly contributes to the performance gap between SNNs and traditional ANNs. To address this issue, we have designed a Spiking Multiscale Attention (SMA) module that captures multiscale spatiotemporal interaction information. Furthermore, we developed a regularization method named Attention ZoneOut (AZO), which utilizes spatiotemporal attention weights to reduce the model's generalization error through pseudo-ensemble training. Our approach has achieved state-of-the-art results on mainstream neural morphology datasets. Additionally, we have reached a performance of 77.1% on the Imagenet-1K dataset using a 104-layer ResNet architecture enhanced with SMA and AZO. This achievement confirms the state-of-the-art performance of SNNs with non-transformer architectures and underscores the effectiveness of our method in bridging the performance gap between SNN models and traditional ANN models.