Desire Backpropagation: A Lightweight Training Algorithm for Multi-Layer Spiking Neural Networks based on Spike-Timing-Dependent Plasticity

TL;DR

This paper introduces desire backpropagation, a novel spike-based supervised learning algorithm for multi-layer spiking neural networks that efficiently minimizes global error using local STDP updates, suitable for low-resource devices.

Contribution

The paper proposes desire backpropagation, a new method to derive desired spike activity from output error, enabling effective training of SNNs with reduced computational complexity.

Findings

Achieved 98.41% accuracy on MNIST classification.

Achieved 87.56% accuracy on Fashion-MNIST classification.

Reduced backward pass complexity by eliminating a multiplication operation.

Abstract

Spiking neural networks (SNNs) are a viable alternative to conventional artificial neural networks when resource efficiency and computational complexity are of importance. A major advantage of SNNs is their binary information transfer through spike trains which eliminates multiplication operations. The training of SNNs has, however, been a challenge, since neuron models are non-differentiable and traditional gradient-based backpropagation algorithms cannot be applied directly. Furthermore, spike-timing-dependent plasticity (STDP), albeit being a spike-based learning rule, updates weights locally and does not optimize for the output error of the network. We present desire backpropagation, a method to derive the desired spike activity of all neurons, including the hidden ones, from the output error. By incorporating this desire value into the local STDP weight update, we can efficiently capture the neuron dynamics while minimizing the global error and attaining a high classification accuracy. That makes desire backpropagation a spike-based supervised learning rule. We trained three-layer networks to classify MNIST and Fashion-MNIST images and reached an accuracy of 98.41% and 87.56%, respectively. In addition, by eliminating a multiplication during the backward pass, we reduce computational complexity and balance arithmetic resources between forward and backward pass, making desire backpropagation a candidate for training on low-resource devices.