Temporal Knowledge Sharing enable Spiking Neural Network Learning from Past and Future

TL;DR

This paper introduces Temporal Knowledge Sharing (TKS), a novel method for Spiking Neural Networks that improves temporal information utilization, reduces latency, and enhances adaptability by enabling learning from past and future information.

Contribution

The paper proposes TKS, a new approach that treats SNNs as temporal aggregation models, allowing effective information sharing across time points and improving performance and flexibility.

Findings

TKS achieves state-of-the-art results on multiple datasets.

TKS enhances temporal generalization and reduces latency.

TKS improves information processing efficiency in SNNs.

Abstract

Spiking Neural Networks (SNNs) have attracted significant attention from researchers across various domains due to their brain-like information processing mechanism. However, SNNs typically grapple with challenges such as extended time steps, low temporal information utilization, and the requirement for consistent time step between testing and training. These challenges render SNNs with high latency. Moreover, the constraint on time steps necessitates the retraining of the model for new deployments, reducing adaptability. To address these issues, this paper proposes a novel perspective, viewing the SNN as a temporal aggregation model. We introduce the Temporal Knowledge Sharing (TKS) method, facilitating information interact between different time points. TKS can be perceived as a form of temporal self-distillation. To validate the efficacy of TKS in information processing, we tested it on static datasets like CIFAR10, CIFAR100, ImageNet-1k, and neuromorphic datasets such as DVS-CIFAR10 and NCALTECH101. Experimental results demonstrate that our method achieves state-of-the-art performance compared to other algorithms. Furthermore, TKS addresses the temporal consistency challenge, endowing the model with superior temporal generalization capabilities. This allows the network to train with longer time steps and maintain high performance during testing with shorter time steps. Such an approach considerably accelerates the deployment of SNNs on edge devices. Finally, we conducted ablation experiments and tested TKS on fine-grained tasks, with results showcasing TKS's enhanced capability to process information efficiently.