Training Deep Spiking Neural Networks using Backpropagation

TL;DR

This paper presents a new backpropagation-based training method for deep spiking neural networks that directly utilizes spike signals and membrane potentials, improving training accuracy and efficiency.

Contribution

Introduces a novel differentiable approach treating membrane potentials as signals, enabling direct backpropagation in deep SNNs, surpassing previous indirect methods.

Findings

Outperforms previous SNN training methods on MNIST benchmarks.

Achieves superior accuracy on event-based N-MNIST dataset.

Enables direct training of deep SNNs with backpropagation.

Abstract

Deep spiking neural networks (SNNs) hold great potential for improving the latency and energy efficiency of deep neural networks through event-based computation. However, training such networks is difficult due to the non-differentiable nature of asynchronous spike events. In this paper, we introduce a novel technique, which treats the membrane potentials of spiking neurons as differentiable signals, where discontinuities at spike times are only considered as noise. This enables an error backpropagation mechanism for deep SNNs, which works directly on spike signals and membrane potentials. Thus, compared with previous methods relying on indirect training and conversion, our technique has the potential to capture the statics of spikes more precisely. Our novel framework outperforms all previously reported results for SNNs on the permutation invariant MNIST benchmark, as well as the N-MNIST benchmark recorded with event-based vision sensors.