Noise Adaptor: Enhancing Low-Latency Spiking Neural Networks through Noise-Injected Low-Bit ANN Conversion

TL;DR

Noise Adaptor introduces a noise-injected training method for low-bit ANNs that significantly improves the accuracy of converted low-latency SNNs without requiring run-time noise correction, enabling deeper architectures like ResNet-101 and ResNet-152.

Contribution

It proposes a novel noise injection technique during ANN training that enhances SNN conversion accuracy and supports deeper network architectures without run-time modifications.

Findings

Achieved competitive accuracy on CIFAR-10 and ImageNet.

Successfully converted ResNet-101 and ResNet-152 to SNNs.

Improved SNN performance without run-time noise correction.

Abstract

We present Noise Adaptor, a novel method for constructing competitive low-latency spiking neural networks (SNNs) by converting noise-injected, low-bit artificial neural networks (ANNs). This approach builds on existing ANN-to-SNN conversion techniques but offers several key improvements: (1) By injecting noise during quantized ANN training, Noise Adaptor better accounts for the dynamic differences between ANNs and SNNs, significantly enhancing SNN accuracy. (2) Unlike previous methods, Noise Adaptor does not require the application of run-time noise correction techniques in SNNs, thereby avoiding modifications to the spiking neuron model and control flow during inference. (3) Our method extends the capability of handling deeper architectures, achieving successful conversions of activation-quantized ResNet-101 and ResNet-152 to SNNs. We demonstrate the effectiveness of our method on CIFAR-10 and ImageNet, achieving competitive performance. The code will be made available as open-source.