Sparse Compressed Spiking Neural Network Accelerator for Object Detection

TL;DR

This paper introduces a sparse compressed SNN accelerator that leverages activation and weight sparsity for efficient object detection, achieving high accuracy and energy efficiency on high-resolution video data.

Contribution

It proposes a novel gated one-to-all product method and a specialized accelerator design to enhance low-power, high-parallelism inference for high-resolution SNN-based object detection.

Findings

Achieves 71.5% mAP on IVS 3cls dataset.

Runs at 1024x576@29 fps with 35.88 TOPS/W energy efficiency.

Consumes 1.05 mJ per frame.

Abstract

Spiking neural networks (SNNs), which are inspired by the human brain, have recently gained popularity due to their relatively simple and low-power hardware for transmitting binary spikes and highly sparse activation maps. However, because SNNs contain extra time dimension information, the SNN accelerator will require more buffers and take longer to infer, especially for the more difficult high-resolution object detection task. As a result, this paper proposes a sparse compressed spiking neural network accelerator that takes advantage of the high sparsity of activation maps and weights by utilizing the proposed gated one-to-all product for low power and highly parallel model execution. The experimental result of the neural network shows 71.5$\%$ mAP with mixed (1,3) time steps on the IVS 3cls dataset. The accelerator with the TSMC 28nm CMOS process can achieve 1024$\times$576@29 frames per second processing when running at 500MHz with 35.88TOPS/W energy efficiency and 1.05mJ energy consumption per frame.