Neurogenesis Dynamics-inspired Spiking Neural Network Training Acceleration
Shaoyi Huang, Haowen Fang, Kaleel Mahmood, Bowen Lei, Nuo Xu, Bin Lei,, Yue Sun, Dongkuan Xu, Wujie Wen, Caiwen Ding

TL;DR
This paper introduces NDSNN, a training acceleration framework for Spiking Neural Networks inspired by neurogenesis dynamics, achieving high sparsity, improved accuracy, and reduced training costs on standard datasets.
Contribution
The paper proposes a novel neurogenesis-inspired dynamic sparsity training method for SNNs that enhances training efficiency without sacrificing model fidelity.
Findings
Achieves up to 20.52% accuracy improvement on TinyImageNet with 99% sparsity.
Reduces training cost to 40.89% of LTH on ResNet-19.
Maintains high accuracy with extreme sparsity through a new drop-and-grow strategy.
Abstract
Biologically inspired Spiking Neural Networks (SNNs) have attracted significant attention for their ability to provide extremely energy-efficient machine intelligence through event-driven operation and sparse activities. As artificial intelligence (AI) becomes ever more democratized, there is an increasing need to execute SNN models on edge devices. Existing works adopt weight pruning to reduce SNN model size and accelerate inference. However, these methods mainly focus on how to obtain a sparse model for efficient inference, rather than training efficiency. To overcome these drawbacks, in this paper, we propose a Neurogenesis Dynamics-inspired Spiking Neural Network training acceleration framework, NDSNN. Our framework is computational efficient and trains a model from scratch with dynamic sparsity without sacrificing model fidelity. Specifically, we design a new drop-and-grow strategy…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Memory and Neural Computing · Ferroelectric and Negative Capacitance Devices · Neural dynamics and brain function
MethodsPruning
