Efficient Event-based Delay Learning in Spiking Neural Networks

TL;DR

This paper introduces a novel event-based training method for spiking neural networks with delays, enabling exact gradient calculation and improving accuracy and efficiency in temporal tasks.

Contribution

The paper presents the first delay learning algorithm for recurrent SNNs based on EventProp, supporting multiple spikes and demonstrating improved performance and efficiency.

Findings

Supports multiple spikes per neuron

Enhances classification accuracy with delays

Uses less memory and is faster than previous methods

Abstract

Spiking Neural Networks (SNNs) compute using sparse communication and are attracting increased attention as a more energy-efficient alternative to traditional Artificial Neural Networks~(ANNs). While standard ANNs are stateless, spiking neurons are stateful and hence intrinsically recurrent, making them well-suited for spatio-temporal tasks. However, the duration of this intrinsic memory is limited by synaptic and membrane time constants. Delays are a powerful additional mechanism and, in this paper, we propose a novel event-based training method for SNNs with delays, grounded in the EventProp formalism which enables the calculation of exact gradients with respect to weights and delays. Our method supports multiple spikes per neuron and, to the best of our knowledge, is the first delay learning algorithm to be applied to recurrent SNNs. We evaluate our method on a simple sequence detection task, as well as the Yin-Yang, Spiking Heidelberg Digits, Spiking Speech Commands and Braille letter reading datasets, demonstrating that our algorithm can optimise delays from suboptimal initial conditions and enhance classification accuracy compared to architectures without delays. We also find that recurrent delays are particularly beneficial in small networks. Finally, we show that our approach uses less than half the memory of the current state-of-the-art delay-learning method and is up to 26x faster.