Learning Delays Through Gradients and Structure: Emergence of Spatiotemporal Patterns in Spiking Neural Networks

TL;DR

This paper introduces a novel spiking neural network model that learns synaptic delays through two methods, demonstrating that dynamic pruning combined with delay learning preserves spatio-temporal patterns and enhances efficiency in processing temporal data.

Contribution

The paper presents two approaches for learning synaptic delays in SNNs, including a dynamic pruning strategy that outperforms traditional delay learning in sparse networks.

Findings

Dynamic pruning preserves spatio-temporal patterns.

Pruning with DEEP R and RigL improves performance.

Delay learning enhances temporal data processing.

Abstract

We present a Spiking Neural Network (SNN) model that incorporates learnable synaptic delays through two approaches: per-synapse delay learning via Dilated Convolutions with Learnable Spacings (DCLS) and a dynamic pruning strategy that also serves as a form of delay learning. In the latter approach, the network dynamically selects and prunes connections, optimizing the delays in sparse connectivity settings. We evaluate both approaches on the Raw Heidelberg Digits keyword spotting benchmark using Backpropagation Through Time with surrogate gradients. Our analysis of the spatio-temporal structure of synaptic interactions reveals that, after training, excitation and inhibition group together in space and time. Notably, the dynamic pruning approach, which employs DEEP R for connection removal and RigL for reconnection, not only preserves these spatio-temporal patterns but outperforms per-synapse delay learning in sparse networks. Our results demonstrate the potential of combining delay learning with dynamic pruning to develop efficient SNN models for temporal data processing. Moreover, the preservation of spatio-temporal dynamics throughout pruning and rewiring highlights the robustness of these features, providing a solid foundation for future neuromorphic computing applications.