Algorithm and Hardware Design of Discrete-Time Spiking Neural Networks Based on Back Propagation with Binary Activations

TL;DR

This paper introduces a novel back propagation training algorithm for discrete-time spiking neural networks using binary activations, enabling efficient learning and high accuracy on neuromorphic hardware.

Contribution

It proposes two new training algorithms for SNNs with binary activations, addressing the challenge of backpropagation in spiking neural networks.

Findings

Achieved over 98% accuracy on MNIST.

Demonstrated low energy consumption on neuromorphic hardware.

Enabled effective training of SNNs with binary activations.

Abstract

We present a new back propagation based training algorithm for discrete-time spiking neural networks (SNN). Inspired by recent deep learning algorithms on binarized neural networks, binary activation with a straight-through gradient estimator is used to model the leaky integrate-fire spiking neuron, overcoming the difficulty in training SNNs using back propagation. Two SNN training algorithms are proposed: (1) SNN with discontinuous integration, which is suitable for rate-coded input spikes, and (2) SNN with continuous integration, which is more general and can handle input spikes with temporal information. Neuromorphic hardware designed in 40nm CMOS exploits the spike sparsity and demonstrates high classification accuracy (>98% on MNIST) and low energy (48.4-773 nJ/image).