Three factor delay learning rules for spiking neural networks

TL;DR

This paper introduces a novel online learning method for spiking neural networks that incorporates synaptic and axonal delays, significantly improving accuracy and efficiency for temporal pattern recognition tasks.

Contribution

It proposes three-factor learning rules for jointly learning delays and weights in SNNs, enabling real-time, resource-efficient training with improved performance.

Findings

Delays improve accuracy by up to 20% over weights-only models.

Joint delay and weight learning yields up to 14% higher accuracy.

Achieves similar accuracy to offline methods on speech recognition dataset, with reduced model size and latency.

Abstract

Spiking Neural Networks (SNNs) are dynamical systems that operate on spatiotemporal data, yet their learnable parameters are often limited to synaptic weights, contributing little to temporal pattern recognition. Learnable parameters that delay spike times can improve classification performance in temporal tasks, but existing methods rely on large networks and offline learning, making them unsuitable for real-time operation in resource-constrained environments. In this paper, we introduce synaptic and axonal delays to leaky integrate and fire (LIF)-based feedforward and recurrent SNNs, and propose three-factor learning rules to simultaneously learn delay parameters online. We employ a smooth Gaussian surrogate to approximate spike derivatives exclusively for the eligibility trace calculation, and together with a top-down error signal determine parameter updates. Our experiments show that incorporating delays improves accuracy by up to 20% over a weights-only baseline, and for networks with similar parameter counts, jointly learning weights and delays yields up to 14% higher accuracy. On the SHD speech recognition dataset, our method achieves similar accuracy to offline backpropagation-based approaches. Compared to state-of-the-art methods, it reduces model size by 6.6x and inference latency by 67%, with only a 2.4% drop in classification accuracy. Our findings benefit the design of power and area-constrained neuromorphic processors by enabling on-device learning and lowering memory requirements.