SpikingGamma: Surrogate-Gradient Free and Temporally Precise Online Training of Spiking Neural Networks with Smoothed Delays

TL;DR

SpikingGamma introduces a novel online training method for spiking neural networks that achieves precise temporal pattern learning without surrogate gradients, enabling efficient hardware implementation and high accuracy on complex tasks.

Contribution

It presents a new spiking neuron model with recursive memory and sigma-delta coding that supports direct error backpropagation, overcoming limitations of surrogate-gradient methods.

Findings

Supports direct error backpropagation without surrogate gradients

Learns fine temporal patterns with minimal spiking in an online manner

Scales to complex tasks with competitive accuracy

Abstract

Neuromorphic hardware implementations of Spiking Neural Networks (SNNs) promise energy-efficient, low-latency AI through sparse, event-driven computation. Yet, training SNNs under fine temporal discretization remains a major challenge, hindering both low-latency responsiveness and the mapping of software-trained SNNs to efficient hardware. In current approaches, spiking neurons are modeled as self-recurrent units, embedded into recurrent networks to maintain state over time, and trained with BPTT or RTRL variants based on surrogate gradients. These methods scale poorly with temporal resolution, while online approximations often exhibit instability for long sequences and tend to fail at capturing temporal patterns precisely. To address these limitations, we develop spiking neurons with internal recursive memory structures that we combine with sigma-delta spike-coding. We show that this SpikingGamma model supports direct error backpropagation without surrogate gradients, can learn fine temporal patterns with minimal spiking in an online manner, and scale feedforward SNNs to complex tasks and benchmarks with competitive accuracy, all while being insensitive to the temporal resolution of the model. Our approach offers both an alternative to current recurrent SNNs trained with surrogate gradients, and a direct route for mapping SNNs to neuromorphic hardware.