BioGrad: Biologically Plausible Gradient-Based Learning for Spiking Neural Networks

TL;DR

This paper introduces a biologically plausible gradient-based learning algorithm for spiking neural networks that matches backpropagation performance while adhering to neuromorphic principles, enabling energy-efficient training on neuromorphic hardware.

Contribution

The authors propose a novel learning algorithm for SNN that is both functionally equivalent to backpropagation and biologically plausible, using local eligibility traces and sleep phases.

Findings

Achieved 98.13% on MNIST with fully connected SNNs.

Deployed on Intel Loihi, trained MNIST with 93.32% accuracy, 400x less energy.

Matched backpropagation performance while maintaining biological plausibility.

Abstract

Spiking neural networks (SNN) are delivering energy-efficient, massively parallel, and low-latency solutions to AI problems, facilitated by the emerging neuromorphic chips. To harness these computational benefits, SNN need to be trained by learning algorithms that adhere to brain-inspired neuromorphic principles, namely event-based, local, and online computations. Yet, the state-of-the-art SNN training algorithms are based on backprop that does not follow the above principles. Due to its limited biological plausibility, the application of backprop to SNN requires non-local feedback pathways for transmitting continuous-valued errors, and relies on gradients from future timesteps. The introduction of biologically plausible modifications to backprop has helped overcome several of its limitations, but limits the degree to which backprop is approximated, which hinders its performance. We propose a biologically plausible gradient-based learning algorithm for SNN that is functionally equivalent to backprop, while adhering to all three neuromorphic principles. We introduced multi-compartment spiking neurons with local eligibility traces to compute the gradients required for learning, and a periodic "sleep" phase to further improve the approximation to backprop during which a local Hebbian rule aligns the feedback and feedforward weights. Our method achieved the same level of performance as backprop with multi-layer fully connected SNN on MNIST (98.13%) and the event-based N-MNIST (97.59%) datasets. We deployed our learning algorithm on Intel's Loihi to train a 1-hidden-layer network for MNIST, and obtained 93.32% test accuracy while consuming 400 times less energy per training sample than BioGrad on GPU. Our work shows that optimal learning is feasible in neuromorphic computing, and further pursuing its biological plausibility can better capture the benefits of this emerging computing paradigm.