A Latency Coding Framework for Deep Spiking Neural Networks with Ultra-Low Latency

TL;DR

This paper introduces a comprehensive training framework for deep spiking neural networks using latency coding, achieving ultra-low latency, high accuracy, and energy efficiency through novel encoding, relaxation of constraints, and adaptive loss functions.

Contribution

It presents a novel latency coding framework with backpropagation through time, feature extraction, relaxed spike constraints, and adaptive loss, enabling efficient training of deep TTFS-coded SNNs.

Findings

Achieves state-of-the-art accuracy with ultra-low inference latency.

Demonstrates improved energy efficiency over existing TTFS models.

Shows enhanced robustness against input corruptions.

Abstract

Spiking neural networks (SNNs) offer a biologically inspired computing paradigm with significant potential for energy-efficient neural processing. Among neural coding schemes of SNNs, Time-To-First-Spike (TTFS) coding, which encodes information through the precise timing of a neuron's first spike, provides exceptional energy efficiency and biological plausibility. Despite its theoretical advantages, existing TTFS models lack efficient training methods, suffering from high inference latency and limited performance. In this work, we present a comprehensive framework, which enables the efficient training of deep TTFS-coded SNNs by employing backpropagation throuh time (BPTT) algorithm. We name the generalized TTFS coding method in our framework as latency coding. The framework includes: (1) a latency encoding (LE) module with feature extraction and straight-through estimators to address severe information loss in direct intensity-to-latency mapping and ensure smooth gradient flow; (2) relaxation of the strict single-spike constraint of traditional TTFS, allowing neurons of intermediate layers to fire multiple times to mitigating gradient vanishing in deep networks; (3) a temporal adaptive decision (TAD) loss function that dynamically weights supervision signals based on sample-dependent confidence, resolving the incompatibility between latency coding and standard cross-entropy loss. Experimental results demonstrate that our method achieves state-of-the-art accuracy in comparison to existing TTFS-coded SNNs with ultra-low inference latency and superior energy efficiency. The framework also demonstrates improved robustness against input corruptions. Our study investigates the characteristics and potential of latency coding in scenarios demanding rapid response, providing valuable insights for further exploiting the temporal learning capabilities of SNNs.