Towards Low-latency Event-based Visual Recognition with Hybrid Step-wise Distillation Spiking Neural Networks

TL;DR

This paper introduces Hybrid Step-wise Distillation, a method to improve low-latency event-based visual recognition with spiking neural networks by balancing accuracy and latency through innovative training and inference strategies.

Contribution

The paper proposes a novel Hybrid Step-wise Distillation approach that disentangles event frame dependency and employs step-wise knowledge distillation to enhance SNN performance at low latency.

Findings

Improved classification accuracy at lower time steps.

Effective reduction of latency without sacrificing performance.

Competitive results on neuromorphic datasets.

Abstract

Spiking neural networks (SNNs) have garnered significant attention for their low power consumption and high biological interpretability. Their rich spatio-temporal information processing capability and event-driven nature make them ideally well-suited for neuromorphic datasets. However, current SNNs struggle to balance accuracy and latency in classifying these datasets. In this paper, we propose Hybrid Step-wise Distillation (HSD) method, tailored for neuromorphic datasets, to mitigate the notable decline in performance at lower time steps. Our work disentangles the dependency between the number of event frames and the time steps of SNNs, utilizing more event frames during the training stage to improve performance, while using fewer event frames during the inference stage to reduce latency. Nevertheless, the average output of SNNs across all time steps is susceptible to individual time step with abnormal outputs, particularly at extremely low time steps. To tackle this issue, we implement Step-wise Knowledge Distillation (SKD) module that considers variations in the output distribution of SNNs at each time step. Empirical evidence demonstrates that our method yields competitive performance in classification tasks on neuromorphic datasets, especially at lower time steps. Our code will be available at: {https://github.com/hsw0929/HSD}.