Neuromorphic Online Learning for Spatiotemporal Patterns with a Forward-only Timeline

TL;DR

This paper introduces SOLSA, an online learning algorithm for spiking neural networks that efficiently learns synaptic weights and temporal filters, outperforming traditional BPTT in accuracy and memory usage.

Contribution

The paper presents SOLSA, a novel online learning method for SNNs with LIF neurons, incorporating synaptic and temporal filter adaptation, reducing memory needs and improving performance.

Findings

SOLSA improves learning accuracy by 14.2% over other non-BPTT methods.

SOLSA achieves 5% higher accuracy than BPTT with 72% less memory.

Enhanced techniques like scheduled updates and early stopping speed convergence.

Abstract

Spiking neural networks (SNNs) are bio-plausible computing models with high energy efficiency. The temporal dynamics of neurons and synapses enable them to detect temporal patterns and generate sequences. While Backpropagation Through Time (BPTT) is traditionally used to train SNNs, it is not suitable for online learning of embedded applications due to its high computation and memory cost as well as extended latency. Previous works have proposed online learning algorithms, but they often utilize highly simplified spiking neuron models without synaptic dynamics and reset feedback, resulting in subpar performance. In this work, we present Spatiotemporal Online Learning for Synaptic Adaptation (SOLSA), specifically designed for online learning of SNNs composed of Leaky Integrate and Fire (LIF) neurons with exponentially decayed synapses and soft reset. The algorithm not only learns the synaptic weight but also adapts the temporal filters associated to the synapses. Compared to the BPTT algorithm, SOLSA has much lower memory requirement and achieves a more balanced temporal workload distribution. Moreover, SOLSA incorporates enhancement techniques such as scheduled weight update, early stop training and adaptive synapse filter, which speed up the convergence and enhance the learning performance. When compared to other non-BPTT based SNN learning, SOLSA demonstrates an average learning accuracy improvement of 14.2%. Furthermore, compared to BPTT, SOLSA achieves a 5% higher average learning accuracy with a 72% reduction in memory cost.