Temporal-adaptive Weight Quantization for Spiking Neural Networks

TL;DR

This paper introduces Temporal-adaptive Weight Quantization (TaWQ), a novel method inspired by biological synaptic modulation, that adaptively quantizes weights in spiking neural networks along the temporal dimension, achieving high energy efficiency with minimal accuracy loss.

Contribution

We propose TaWQ, a new quantization technique that incorporates temporal dynamics to improve weight quantization in SNNs, inspired by biological neural mechanisms.

Findings

Maintains high energy efficiency with 4.12M, 0.63mJ energy consumption.

Achieves only 0.22% accuracy loss on ImageNet.

Effective on both static and neuromorphic datasets.

Abstract

Weight quantization in spiking neural networks (SNNs) could further reduce energy consumption. However, quantizing weights without sacrificing accuracy remains challenging. In this study, inspired by astrocyte-mediated synaptic modulation in the biological nervous systems, we propose Temporal-adaptive Weight Quantization (TaWQ), which incorporates weight quantization with temporal dynamics to adaptively allocate ultra-low-bit weights along the temporal dimension. Extensive experiments on static (e.g., ImageNet) and neuromorphic (e.g., CIFAR10-DVS) datasets demonstrate that our TaWQ maintains high energy efficiency (4.12M, 0.63mJ) while incurring a negligible quantization loss of only 0.22% on ImageNet.