Hoyer regularizer is all you need for ultra low-latency spiking neural networks

TL;DR

This paper introduces a novel training framework for ultra low-latency spiking neural networks using a Hoyer regularizer, enabling effective one-time-step SNNs with improved accuracy and efficiency for vision tasks.

Contribution

The paper proposes a new method for training one-time-step SNNs with a Hoyer regularizer that estimates layer thresholds, reducing complexity and improving performance.

Findings

Outperforms existing SNNs in accuracy-FLOPs trade-off

Enables effective one-time-step SNN training

Demonstrates success on image recognition and object detection

Abstract

Spiking Neural networks (SNN) have emerged as an attractive spatio-temporal computing paradigm for a wide range of low-power vision tasks. However, state-of-the-art (SOTA) SNN models either incur multiple time steps which hinder their deployment in real-time use cases or increase the training complexity significantly. To mitigate this concern, we present a training framework (from scratch) for one-time-step SNNs that uses a novel variant of the recently proposed Hoyer regularizer. We estimate the threshold of each SNN layer as the Hoyer extremum of a clipped version of its activation map, where the clipping threshold is trained using gradient descent with our Hoyer regularizer. This approach not only downscales the value of the trainable threshold, thereby emitting a large number of spikes for weight update with a limited number of iterations (due to only one time step) but also shifts the membrane potential values away from the threshold, thereby mitigating the effect of noise that can degrade the SNN accuracy. Our approach outperforms existing spiking, binary, and adder neural networks in terms of the accuracy-FLOPs trade-off for complex image recognition tasks. Downstream experiments on object detection also demonstrate the efficacy of our approach.