Spiking Deep Networks with LIF Neurons

TL;DR

This paper demonstrates that biologically-plausible spiking deep networks with LIF neurons can achieve state-of-the-art results on CIFAR-10 and MNIST by novel training techniques that improve robustness and performance.

Contribution

It introduces a method to train LIF neuron-based deep networks effectively, achieving high performance and biological plausibility for neuromorphic hardware applications.

Findings

Achieved state-of-the-art results on CIFAR-10 and MNIST datasets.

Developed a training method with bounded derivative and noise robustness.

Demonstrated applicability to neuromorphic hardware.

Abstract

We train spiking deep networks using leaky integrate-and-fire (LIF) neurons, and achieve state-of-the-art results for spiking networks on the CIFAR-10 and MNIST datasets. This demonstrates that biologically-plausible spiking LIF neurons can be integrated into deep networks can perform as well as other spiking models (e.g. integrate-and-fire). We achieved this result by softening the LIF response function, such that its derivative remains bounded, and by training the network with noise to provide robustness against the variability introduced by spikes. Our method is general and could be applied to other neuron types, including those used on modern neuromorphic hardware. Our work brings more biological realism into modern image classification models, with the hope that these models can inform how the brain performs this difficult task. It also provides new methods for training deep networks to run on neuromorphic hardware, with the aim of fast, power-efficient image classification for robotics applications.