SPICEprop: Backpropagating Errors Through Memristive Spiking Neural Networks

TL;DR

This paper introduces a fully memristive spiking neural network trained with backpropagation through time, achieving high accuracy on MNIST and Fashion-MNIST datasets by leveraging differentiable memristive neuron models.

Contribution

It presents a novel memristive neuron model and training method that enables direct gradient-based learning in fully memristive spiking neural networks.

Findings

Achieved 97.58% accuracy on MNIST

Achieved 75.26% accuracy on Fashion-MNIST

First fully memristive SNN with backpropagation trained on SPICE models

Abstract

We present a fully memristive spiking neural network (MSNN) consisting of novel memristive neurons trained using the backpropagation through time (BPTT) learning rule. Gradient descent is applied directly to the memristive integrated-and-fire (MIF) neuron designed using analog SPICE circuit models, which generates distinct depolarization, hyperpolarization, and repolarization voltage waveforms. Synaptic weights are trained by BPTT using the membrane potential of the MIF neuron model and can be processed on memristive crossbars. The natural spiking dynamics of the MIF neuron model are fully differentiable, eliminating the need for gradient approximations that are prevalent in the spiking neural network literature. Despite the added complexity of training directly on SPICE circuit models, we achieve 97.58% accuracy on the MNIST testing dataset and 75.26% on the Fashion-MNIST testing dataset, the highest accuracies among all fully MSNNs.