Training Robust Spiking Neural Networks on Neuromorphic Data with Spatiotemporal Fragments

TL;DR

This paper introduces a novel data augmentation method called Event SpatioTemporal Fragments (ESTF) for training robust spiking neural networks on neuromorphic vision data, significantly improving accuracy and robustness.

Contribution

The paper proposes ESTF, a new augmentation technique that preserves data continuity and enhances SNN robustness on neuromorphic datasets.

Findings

ESTF outperforms other augmentation methods.

Achieves state-of-the-art 83.9% accuracy on CIFAR10-DVS.

Improves robustness of SNNs to brightness disturbances.

Abstract

Neuromorphic vision sensors (event cameras) are inherently suitable for spiking neural networks (SNNs) and provide novel neuromorphic vision data for this biomimetic model. Due to the spatiotemporal characteristics, novel data augmentations are required to process the unconventional visual signals of these cameras. In this paper, we propose a novel Event SpatioTemporal Fragments (ESTF) augmentation method. It preserves the continuity of neuromorphic data by drifting or inverting fragments of the spatiotemporal event stream to simulate the disturbance of brightness variations, leading to more robust spiking neural networks. Extensive experiments are performed on prevailing neuromorphic datasets. It turns out that ESTF provides substantial improvements over pure geometric transformations and outperforms other event data augmentation methods. It is worth noting that the SNNs with ESTF achieve the state-of-the-art accuracy of 83.9\% on the CIFAR10-DVS dataset.