Fast and Efficient Asynchronous Neural Computation with Adapting Spiking Neural Networks

TL;DR

This paper introduces an adaptive spiking neural network (ASNN) that efficiently encodes information with fewer spikes, enabling fast, asynchronous computation comparable to traditional neural networks, with improved speed and reduced spike usage.

Contribution

The paper presents a novel ASNN model based on adaptive neurons that encode information efficiently and can replace ReLU units in deep networks, outperforming existing SNNs in speed and spike efficiency.

Findings

ASNN performs comparably to artificial neural networks in classification tasks.

ASNN responds up to ten times faster than current SNNs.

ASNN uses an order of magnitude fewer spikes and network updates.

Abstract

Biological neurons communicate with a sparing exchange of pulses - spikes. It is an open question how real spiking neurons produce the kind of powerful neural computation that is possible with deep artificial neural networks, using only so very few spikes to communicate. Building on recent insights in neuroscience, we present an Adapting Spiking Neural Network (ASNN) based on adaptive spiking neurons. These spiking neurons efficiently encode information in spike-trains using a form of Asynchronous Pulsed Sigma-Delta coding while homeostatically optimizing their firing rate. In the proposed paradigm of spiking neuron computation, neural adaptation is tightly coupled to synaptic plasticity, to ensure that downstream neurons can correctly decode upstream spiking neurons. We show that this type of network is inherently able to carry out asynchronous and event-driven neural computation, while performing identical to corresponding artificial neural networks (ANNs). In particular, we show that these adaptive spiking neurons can be drop in replacements for ReLU neurons in standard feedforward ANNs comprised of such units. We demonstrate that this can also be successfully applied to a ReLU based deep convolutional neural network for classifying the MNIST dataset. The ASNN thus outperforms current Spiking Neural Networks (SNNs) implementations, while responding (up to) an order of magnitude faster and using an order of magnitude fewer spikes. Additionally, in a streaming setting where frames are continuously classified, we show that the ASNN requires substantially fewer network updates as compared to the corresponding ANN.