FireFly-S: Exploiting Dual-Side Sparsity for Spiking Neural Networks Acceleration with Reconfigurable Spatial Architecture

TL;DR

FireFly-S introduces a co-optimized software-hardware approach that exploits dual-side sparsity in Spiking Neural Networks, achieving high efficiency and reconfigurability on FPGA-based accelerators with minimal accuracy loss.

Contribution

The paper presents a novel framework combining gradient rewiring, modified LSQ, and a spatial architecture to leverage dual-side sparsity for SNN acceleration on FPGAs.

Findings

Achieves over 85% weight sparsity with 4-bit quantization and negligible accuracy loss.

Delivers up to 10,047 FPS/W on MNIST dataset.

Employs a parametric spatial architecture with inter-layer pipelining for flexible deployment.

Abstract

Spiking Neural Networks (SNNs), with brain-inspired structure using discrete spikes instead of continuous activations, are gaining attention for their efficient processing on neuromorphic chips. While current SNN hardware accelerators often prioritize temporal spike sparsity, exploiting sparse synaptic weights offers significant untapped potential for even greater efficiency. To address this, we propose FireFly-S, a Sparse extension of the FireFly series. This co-optimized software-hardware design focuses on leveraging dual-side sparsity for acceleration. On the software side, we propose a algorithmic optimization framework that combines gradient rewiring for pruning and modified Learned Step Size Quantization (LSQ) for SNNs, achieving a weight sparsity exceeding 85\% and enabling efficient 4-bit quantization with negligible accuracy loss. On the hardware side, we present an efficient dual-side sparsity detector employing a Bitmap-based sparse decoding logic to pinpoint the positions of non-zero weights and input spikes. The logic allows for direct bypassing of redundant computations, thereby enhancing computational efficiency. Different from the overlay architecture adopted by previous FireFly series, we adopt a parametric spatial architecture with inter-layer pipelining that can fully exploit the fine-grained programmability and reconfigurability of Field-Programmable Gate Arrays (FPGAs), enabling fast deployment for various models. A spatial-temporal dataflow is also proposed to support such inter-layer pipelining and avoid long-term temporal dependencies. In experiments conducted on the MNIST, DVS-Gesture and CIFAR-10 datasets, the FireFly-S model achieves 85--95\% sparsity with 4-bit quantization and the hardware accelerator effectively leverages the dual-side sparsity, delivering performance metrics of 10,047~FPS/W on MNIST, 3,683~FPS/W on DVS-Gesture, and 2,327~FPS/W on CIFAR-10.