InfLoR-SNN: Reducing Information Loss for Spiking Neural Networks

TL;DR

This paper introduces a novel 'Soft Reset' mechanism and Membrane Potential Rectifier for Spiking Neural Networks, significantly reducing information loss caused by the traditional 'Hard Reset' and quantization errors, leading to improved performance.

Contribution

It proposes a new 'Soft Reset' mechanism and MPR to address information loss in SNNs, enhancing their accuracy on static and dynamic datasets.

Findings

Outperforms vanilla SNNs on static datasets

Achieves better results on dynamic datasets

Reduces information loss with new mechanisms

Abstract

The Spiking Neural Network (SNN) has attracted more and more attention recently. It adopts binary spike signals to transmit information. Benefitting from the information passing paradigm of SNNs, the multiplications of activations and weights can be replaced by additions, which are more energy-efficient. However, its "Hard Reset" mechanism for the firing activity would ignore the difference among membrane potentials when the membrane potential is above the firing threshold, causing information loss. Meanwhile, quantifying the membrane potential to 0/1 spikes at the firing instants will inevitably introduce the quantization error thus bringing about information loss too. To address these problems, we propose to use the "Soft Reset" mechanism for the supervised training-based SNNs, which will drive the membrane potential to a dynamic reset potential according to its magnitude, and Membrane Potential Rectifier (MPR) to reduce the quantization error via redistributing the membrane potential to a range close to the spikes. Results show that the SNNs with the "Soft Reset" mechanism and MPR outperform their vanilla counterparts on both static and dynamic datasets.