GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks

TL;DR

GLIF introduces a unified, learnable spiking neuron model that combines multiple biological features, enhancing the expressiveness and performance of SNNs, demonstrated by state-of-the-art results on CIFAR-100.

Contribution

The paper proposes GLIF, a novel unified neuron model that fuses multiple bio-features with learnable gating, increasing heterogeneity and adaptability in SNNs.

Findings

GLIF outperforms existing SNNs on various datasets.

Achieved 77.35% top-1 accuracy on CIFAR-100 with a spiking ResNet-19.

GLIF enhances neuron diversity and network performance.

Abstract

Spiking Neural Networks (SNNs) have been studied over decades to incorporate their biological plausibility and leverage their promising energy efficiency. Throughout existing SNNs, the leaky integrate-and-fire (LIF) model is commonly adopted to formulate the spiking neuron and evolves into numerous variants with different biological features. However, most LIF-based neurons support only single biological feature in different neuronal behaviors, limiting their expressiveness and neuronal dynamic diversity. In this paper, we propose GLIF, a unified spiking neuron, to fuse different bio-features in different neuronal behaviors, enlarging the representation space of spiking neurons. In GLIF, gating factors, which are exploited to determine the proportion of the fused bio-features, are learnable during training. Combining all learnable membrane-related parameters, our method can make spiking neurons different and constantly changing, thus increasing the heterogeneity and adaptivity of spiking neurons. Extensive experiments on a variety of datasets demonstrate that our method obtains superior performance compared with other SNNs by simply changing their neuronal formulations to GLIF. In particular, we train a spiking ResNet-19 with GLIF and achieve $77.35\%$ top-1 accuracy with six time steps on CIFAR-100, which has advanced the state-of-the-art. Codes are available at \url{https://github.com/Ikarosy/Gated-LIF}.