Long Short-term Memory with Two-Compartment Spiking Neuron

TL;DR

This paper introduces a biologically inspired LSTM-LIF spiking neuron model with two compartments, enhancing long-term memory retention and addressing vanishing gradients, leading to improved temporal classification and energy efficiency in neuromorphic systems.

Contribution

The paper presents a novel two-compartment spiking neuron model that improves long-term memory and training efficiency for temporal tasks, inspired by biological neurons.

Findings

Superior temporal classification performance

Rapid convergence during training

High energy efficiency

Abstract

The identification of sensory cues associated with potential opportunities and dangers is frequently complicated by unrelated events that separate useful cues by long delays. As a result, it remains a challenging task for state-of-the-art spiking neural networks (SNNs) to identify long-term temporal dependencies since bridging the temporal gap necessitates an extended memory capacity. To address this challenge, we propose a novel biologically inspired Long Short-Term Memory Leaky Integrate-and-Fire spiking neuron model, dubbed LSTM-LIF. Our model incorporates carefully designed somatic and dendritic compartments that are tailored to retain short- and long-term memories. The theoretical analysis further confirms its effectiveness in addressing the notorious vanishing gradient problem. Our experimental results, on a diverse range of temporal classification tasks, demonstrate superior temporal classification capability, rapid training convergence, strong network generalizability, and high energy efficiency of the proposed LSTM-LIF model. This work, therefore, opens up a myriad of opportunities for resolving challenging temporal processing tasks on emerging neuromorphic computing machines.