Optimization of Low-Latency Spiking Neural Networks Utilizing Historical Dynamics of Refractory Periods

TL;DR

This paper introduces a dynamic refractory period model for low-latency spiking neural networks that improves noise resistance and reduces redundant spikes, achieving state-of-the-art accuracy on various datasets.

Contribution

The paper proposes a novel HDRP model that dynamically adjusts refractory periods based on membrane potential derivatives, enhancing robustness and performance in low-latency SNNs.

Findings

HDRP-SNN reduces redundant spikes significantly.

HDRP-SNN achieves SOTA accuracy on static and neuromorphic datasets.

HDRP-SNN outperforms traditional SNNs and ANNs in noise resistance.

Abstract

The refractory period controls neuron spike firing rate, crucial for network stability and noise resistance. With advancements in spiking neural network (SNN) training methods, low-latency SNN applications have expanded. In low-latency SNNs, shorter simulation steps render traditional refractory mechanisms, which rely on empirical distributions or spike firing rates, less effective. However, omitting the refractory period amplifies the risk of neuron over-activation and reduces the system's robustness to noise. To address this challenge, we propose a historical dynamic refractory period (HDRP) model that leverages membrane potential derivative with historical refractory periods to estimate an initial refractory period and dynamically adjust its duration. Additionally, we propose a threshold-dependent refractory kernel to mitigate excessive neuron state accumulation. Our approach retains the binary characteristics of SNNs while enhancing both noise resistance and overall performance. Experimental results show that HDRP-SNN significantly reduces redundant spikes compared to traditional SNNs, and achieves state-of-the-art (SOTA) accuracy both on static datasets and neuromorphic datasets. Moreover, HDRP-SNN outperforms artificial neural networks (ANNs) and traditional SNNs in noise resistance, highlighting the crucial role of the HDRP mechanism in enhancing the performance of low-latency SNNs.