CLIF: Complementary Leaky Integrate-and-Fire Neuron for Spiking Neural Networks

TL;DR

This paper introduces CLIF, a novel neuron model for spiking neural networks that enhances training by improving gradient flow, leading to superior accuracy and efficiency, sometimes surpassing traditional artificial neural networks.

Contribution

The paper proposes the CLIF neuron, which creates additional pathways for backpropagation in SNNs, addressing training challenges and improving performance without added hyperparameters.

Findings

CLIF outperforms other neuron models in various datasets.

CLIF's performance slightly exceeds that of comparable ANNs.

The model is hyperparameter-free and broadly applicable.

Abstract

Spiking neural networks (SNNs) are promising brain-inspired energy-efficient models. Compared to conventional deep Artificial Neural Networks (ANNs), SNNs exhibit superior efficiency and capability to process temporal information. However, it remains a challenge to train SNNs due to their undifferentiable spiking mechanism. The surrogate gradients method is commonly used to train SNNs, but often comes with an accuracy disadvantage over ANNs counterpart. We link the degraded accuracy to the vanishing of gradient on the temporal dimension through the analytical and experimental study of the training process of Leaky Integrate-and-Fire (LIF) Neuron-based SNNs. Moreover, we propose the Complementary Leaky Integrate-and-Fire (CLIF) Neuron. CLIF creates extra paths to facilitate the backpropagation in computing temporal gradient while keeping binary output. CLIF is hyperparameter-free and features broad applicability. Extensive experiments on a variety of datasets demonstrate CLIF's clear performance advantage over other neuron models. Furthermore, the CLIF's performance even slightly surpasses superior ANNs with identical network structure and training conditions. The code is available at https://github.com/HuuYuLong/Complementary-LIF.