AutoSNN: Towards Energy-Efficient Spiking Neural Networks

TL;DR

AutoSNN introduces a neural architecture search framework that optimizes spiking neural networks for higher accuracy and lower energy consumption by considering both performance and spike activity during the search process.

Contribution

The paper presents AutoSNN, a novel spike-aware neural architecture search method that improves SNN accuracy and energy efficiency by optimizing architecture design choices.

Findings

AutoSNN outperforms hand-crafted SNNs in accuracy.

AutoSNN reduces the number of spikes, enhancing energy efficiency.

Effective on various neuromorphic datasets.

Abstract

Spiking neural networks (SNNs) that mimic information transmission in the brain can energy-efficiently process spatio-temporal information through discrete and sparse spikes, thereby receiving considerable attention. To improve accuracy and energy efficiency of SNNs, most previous studies have focused solely on training methods, and the effect of architecture has rarely been studied. We investigate the design choices used in the previous studies in terms of the accuracy and number of spikes and figure out that they are not best-suited for SNNs. To further improve the accuracy and reduce the spikes generated by SNNs, we propose a spike-aware neural architecture search framework called AutoSNN. We define a search space consisting of architectures without undesirable design choices. To enable the spike-aware architecture search, we introduce a fitness that considers both the accuracy and number of spikes. AutoSNN successfully searches for SNN architectures that outperform hand-crafted SNNs in accuracy and energy efficiency. We thoroughly demonstrate the effectiveness of AutoSNN on various datasets including neuromorphic datasets.