Multi-Bit Mechanism: A Novel Information Transmission Paradigm for Spiking Neural Networks

TL;DR

This paper introduces a multi-bit transmission mechanism for spiking neural networks to improve their expressiveness and performance while maintaining low power consumption, addressing the information loss caused by binary spikes.

Contribution

The paper proposes a novel multi-bit information transmission paradigm for SNNs, enhancing expressiveness and reducing information loss compared to traditional binary spike methods.

Findings

Consistent performance improvements in experiments

Effective utilization of multi-bit spikes for better information transmission

Enhanced neuron stimulation from previous layers

Abstract

Since proposed, spiking neural networks (SNNs) gain recognition for their high performance, low power consumption and enhanced biological interpretability. However, while bringing these advantages, the binary nature of spikes also leads to considerable information loss in SNNs, ultimately causing performance degradation. We claim that the limited expressiveness of current binary spikes, resulting in substantial information loss, is the fundamental issue behind these challenges. To alleviate this, our research introduces a multi-bit information transmission mechanism for SNNs. This mechanism expands the output of spiking neurons from the original single bit to multiple bits, enhancing the expressiveness of the spikes and reducing information loss during the forward process, while still maintaining the low energy consumption advantage of SNNs. For SNNs, this represents a new paradigm of information transmission. Moreover, to further utilize the limited spikes, we extract effective signals from the previous layer to re-stimulate the neurons, thus encouraging full spikes emission across various bit levels. We conducted extensive experiments with our proposed method using both direct training method and ANN-SNN conversion method, and the results show consistent performance improvements.