ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks

TL;DR

This paper introduces ES-ImageNet, a large-scale event-stream dataset for spiking neural networks, created efficiently by converting existing image datasets using a novel algorithm, enabling improved neuromorphic vision research.

Contribution

The paper presents a fast, software-based method to generate a large event-stream dataset from ImageNet, significantly expanding resources for neuromorphic vision research.

Findings

ES-ImageNet contains approximately 1.3 million event-stream samples.

The ODG algorithm efficiently converts images to event streams with high quality.

Benchmark results demonstrate the dataset's utility for SNN and deep neural network training.

Abstract

With event-driven algorithms, especially the spiking neural networks (SNNs), achieving continuous improvement in neuromorphic vision processing, a more challenging event-stream-dataset is urgently needed. However, it is well known that creating an ES-dataset is a time-consuming and costly task with neuromorphic cameras like dynamic vision sensors (DVS). In this work, we propose a fast and effective algorithm termed Omnidirectional Discrete Gradient (ODG) to convert the popular computer vision dataset ILSVRC2012 into its event-stream (ES) version, generating about 1,300,000 frame-based images into ES-samples in 1000 categories. In this way, we propose an ES-dataset called ES-ImageNet, which is dozens of times larger than other neuromorphic classification datasets at present and completely generated by the software. The ODG algorithm implements an image motion to generate local value changes with discrete gradient information in different directions, providing a low-cost and high-speed way for converting frame-based images into event streams, along with Edge-Integral to reconstruct the high-quality images from event streams. Furthermore, we analyze the statistics of the ES-ImageNet in multiple ways, and a performance benchmark of the dataset is also provided using both famous deep neural network algorithms and spiking neural network algorithms. We believe that this work shall provide a new large-scale benchmark dataset for SNNs and neuromorphic vision.