Gated Parametric Neuron for Spike-based Audio Recognition

TL;DR

This paper introduces a gated parametric neuron (GPN) for spike-based audio recognition, addressing vanishing gradients and enabling automatic learning of heterogeneous neuronal parameters, leading to improved SNN performance.

Contribution

The paper proposes the GPN, a novel neuron model that enhances gradient flow and learns diverse neuronal parameters automatically, advancing spike-based neural network capabilities.

Findings

GPN outperforms state-of-the-art SNNs on audio datasets.

GPN mitigates vanishing gradient issues effectively.

GPN learns spatio-temporal heterogeneous parameters.

Abstract

Spiking neural networks (SNNs) aim to simulate real neural networks in the human brain with biologically plausible neurons. The leaky integrate-and-fire (LIF) neuron is one of the most widely studied SNN architectures. However, it has the vanishing gradient problem when trained with backpropagation. Additionally, its neuronal parameters are often manually specified and fixed, in contrast to the heterogeneity of real neurons in the human brain. This paper proposes a gated parametric neuron (GPN) to process spatio-temporal information effectively with the gating mechanism. Compared with the LIF neuron, the GPN has two distinguishing advantages: 1) it copes well with the vanishing gradients by improving the flow of gradient propagation; and, 2) it learns spatio-temporal heterogeneous neuronal parameters automatically. Additionally, we use the same gate structure to eliminate initial neuronal parameter selection and design a hybrid recurrent neural network-SNN structure. Experiments on two spike-based audio datasets demonstrated that the GPN network outperformed several state-of-the-art SNNs, could mitigate vanishing gradients, and had spatio-temporal heterogeneous parameters. Our work shows the ability of SNNs to handle long-term dependencies and achieve high performance simultaneously.