Ternary Spiking Neural Networks Enhanced by Complemented Neurons and Membrane Potential Aggregation

TL;DR

This paper introduces a novel ternary spiking neuron model called Complemented Ternary Spiking Neuron (CTSN) with learnable complemental terms, and a training method named Temporal Membrane Potential Regularization (TMPR), significantly improving biological plausibility and information capacity of SNNs.

Contribution

The paper proposes CTSN with adaptive dynamics and TMPR training, addressing key limitations of existing ternary spiking neurons and enhancing SNN performance.

Findings

CTSN improves information retention and neuron heterogeneity.

TMPR enhances training stability and accuracy.

Experimental results show state-of-the-art performance on multiple datasets.

Abstract

Spiking Neural Networks (SNNs) are promising energy-efficient models and powerful framworks of modeling neuron dynamics. However, existing binary spiking neurons exhibit limited biological plausibilities and low information capacity. Recently developed ternary spiking neuron possesses higher consistency with biological principles (i.e. excitation-inhibition balance mechanism). Despite of this, the ternary spiking neuron suffers from defects including iterative information loss, temporal gradient vanishing and irregular distributions of membrane potentials. To address these issues, we propose Complemented Ternary Spiking Neuron (CTSN), a novel ternary spiking neuron model that incorporates an learnable complemental term to store information from historical inputs. CTSN effectively improves the deficiencies of ternary spiking neuron, while the embedded learnable factors enable CTSN to adaptively adjust neuron dynamics, providing strong neural heterogeneity. Furthermore, based on the temporal evolution features of ternary spiking neurons' membrane potential distributions, we propose the Temporal Membrane Potential Regularization (TMPR) training method. TMPR introduces time-varying regularization strategy utilizing membrane potentials, furhter enhancing the training process by creating extra backpropagation paths. We validate our methods through extensive experiments on various datasets, demonstrating remarkable performance advances.