Is Neuromorphic MNIST neuromorphic? Analyzing the discriminative power of neuromorphic datasets in the time domain

TL;DR

This study evaluates whether N-MNIST is truly neuromorphic by analyzing the information encoded in the time domain and comparing the performance of SNNs and ANNs, concluding that N-MNIST does not showcase SNN advantages.

Contribution

The paper provides the first unsupervised SNN trained on N-MNIST and critically assesses the dataset's ability to demonstrate neuromorphic advantages.

Findings

ANN achieves 99.23% accuracy on N-MNIST

Unsupervised SNN achieves 91.78% accuracy on N-MNIST

Rate-based SNNs outperform timing-based SNNs on N-MNIST

Abstract

The advantage of spiking neural networks (SNNs) over their predecessors is their ability to spike, enabling them to use spike timing for coding and efficient computing. A neuromorphic dataset should allow a neuromorphic algorithm to clearly show that a SNN is able to perform better on the dataset than an ANN. We have analyzed both N-MNIST and N-Caltech101 along these lines, but focus our study on N-MNIST. First we evaluate if additional information is encoded in the time domain in a neuromoprhic dataset. We show that an ANN trained with backpropagation on frame based versions of N-MNIST and N-Caltech101 images achieve 99.23% and 78.01% accuracy. These are the best classification accuracies obtained on these datasets to date. Second we present the first unsupervised SNN to be trained on N-MNIST and demonstrate results of 91.78%. We also use this SNN for further experiments on N-MNIST to show that rate based SNNs perform better, and precise spike timings are not important in N-MNIST. N-MNIST does not, therefore, highlight the unique ability of SNNs. The conclusion of this study opens an important question in neuromorphic engineering - what, then, constitutes a good neuromorphic dataset?