Combining Convolution and Delay Learning in Recurrent Spiking Neural Networks

TL;DR

This paper enhances recurrent spiking neural networks by integrating convolutional recurrent connections with delay learning, resulting in a more efficient architecture with significant memory and speed improvements without sacrificing accuracy.

Contribution

It introduces a novel combination of convolutional recurrent connections with delay learning in SNNs, achieving efficiency gains while maintaining performance.

Findings

99% reduction in recurrent parameters

52x faster inference time

Retains accuracy of previous delay learning methods

Abstract

Spiking neural networks (SNNs) are rapidly gaining momentum as an alternative to conventional artificial neural networks in resource constrained edge systems. In this work, we continue a recent research line on recurrent SNNs where axonal delays are learned at runtime along with the other network parameters. The first proposed approach, dubbed DelRec, demonstrated the benefit of recurrent delay learning in SNNs. Here, we extend it by advocating the use of convolutional recurrent connections in conjunction with the DelRec delay learning mechanism. According to our tests on an audio classification task, this leads to a streamlined architecture with smaller memory footprint (around 99% savings in terms of number of recurrent parameters) and a much faster (52x) inference time, while retaining DelRec's accuracy. Our code is available at: https://github.com/luciozebendo/delrec_snn/tree/conv_delays