TC-LIF: A Two-Compartment Spiking Neuron Model for Long-Term Sequential Modelling

TL;DR

This paper introduces TC-LIF, a biologically inspired two-compartment spiking neuron model designed to improve long-term temporal dependency learning in neural networks, demonstrating superior performance and efficiency on temporal classification tasks.

Contribution

The paper presents a novel two-compartment spiking neuron model, TC-LIF, with theoretical validation and improved capabilities for long-term temporal dependencies in neural networks.

Findings

Superior temporal classification performance

Rapid convergence during training

High energy efficiency

Abstract

The identification of sensory cues associated with potential opportunities and dangers is frequently complicated by unrelated events that separate useful cues by long delays. As a result, it remains a challenging task for state-of-the-art spiking neural networks (SNNs) to establish long-term temporal dependency between distant cues. To address this challenge, we propose a novel biologically inspired Two-Compartment Leaky Integrate-and-Fire spiking neuron model, dubbed TC-LIF. The proposed model incorporates carefully designed somatic and dendritic compartments that are tailored to facilitate learning long-term temporal dependencies. Furthermore, a theoretical analysis is provided to validate the effectiveness of TC-LIF in propagating error gradients over an extended temporal duration. Our experimental results, on a diverse range of temporal classification tasks, demonstrate superior temporal classification capability, rapid training convergence, and high energy efficiency of the proposed TC-LIF model. Therefore, this work opens up a myriad of opportunities for solving challenging temporal processing tasks on emerging neuromorphic computing systems. Our code is publicly available at https://github.com/ZhangShimin1/TC-LIF.