Resource Constrained Model Compression via Minimax Optimization for Spiking Neural Networks

TL;DR

This paper introduces a minimax optimization approach for compressing Spiking Neural Networks, achieving state-of-the-art results while balancing performance and resource efficiency on edge devices.

Contribution

It presents an end-to-end minimax optimization method for SNN compression, improving over existing pruning and sparse learning techniques.

Findings

Achieved state-of-the-art performance on benchmark datasets.

Joint compression and finetuning outperform sequential methods.

Effective balance between model accuracy and resource efficiency.

Abstract

Brain-inspired Spiking Neural Networks (SNNs) have the characteristics of event-driven and high energy-efficient, which are different from traditional Artificial Neural Networks (ANNs) when deployed on edge devices such as neuromorphic chips. Most previous work focuses on SNNs training strategies to improve model performance and brings larger and deeper network architectures. It is difficult to deploy these complex networks on resource-limited edge devices directly. To meet such demand, people compress SNNs very cautiously to balance the performance and the computation efficiency. Existing compression methods either iteratively pruned SNNs using weights norm magnitude or formulated the problem as a sparse learning optimization. We propose an improved end-to-end Minimax optimization method for this sparse learning problem to better balance the model performance and the computation efficiency. We also demonstrate that jointly applying compression and finetuning on SNNs is better than sequentially, especially for extreme compression ratios. The compressed SNN models achieved state-of-the-art (SOTA) performance on various benchmark datasets and architectures. Our code is available at https://github.com/chenjallen/Resource-Constrained-Compression-on-SNN.