PMSN: A Parallel Multi-compartment Spiking Neuron for Multi-scale Temporal Processing

TL;DR

This paper introduces PMSN, a novel multi-compartment spiking neuron model that enhances multi-scale temporal processing in SNNs, significantly improving accuracy and speed in pattern recognition tasks.

Contribution

The paper presents PMSN, a biologically inspired neuron model with multiple interacting substructures and parallel training techniques, advancing multi-scale temporal processing in SNNs.

Findings

Outperforms state-of-the-art SNN neurons in temporal processing.

Achieves over 10x simulation acceleration compared to Leaky Integrate-and-Fire.

Improves accuracy by 30% on Sequential CIFAR10.

Abstract

Spiking Neural Networks (SNNs) hold great potential to realize brain-inspired, energy-efficient computational systems. However, current SNNs still fall short in terms of multi-scale temporal processing compared to their biological counterparts. This limitation has resulted in poor performance in many pattern recognition tasks with information that varies across different timescales. To address this issue, we put forward a novel spiking neuron model called Parallel Multi-compartment Spiking Neuron (PMSN). The PMSN emulates biological neurons by incorporating multiple interacting substructures and allows for flexible adjustment of the substructure counts to effectively represent temporal information across diverse timescales. Additionally, to address the computational burden associated with the increased complexity of the proposed model, we introduce two parallelization techniques that decouple the temporal dependencies of neuronal updates, enabling parallelized training across different time steps. Our experimental results on a wide range of pattern recognition tasks demonstrate the superiority of PMSN. It outperforms other state-of-the-art spiking neuron models in terms of its temporal processing capacity, training speed, and computation cost. Specifically, compared with the commonly used Leaky Integrate-and-Fire neuron, PMSN offers a simulation acceleration of over 10 $\times$ and a 30 % improvement in accuracy on Sequential CIFAR10 dataset, while maintaining comparable computational cost.