Activation-wise Propagation: A One-Timestep Strategy for Spiking Neural Networks

TL;DR

This paper introduces AMP2, a novel hidden state update method for SNNs that improves efficiency and accuracy by enabling dynamic membrane potential transmission among neurons, reducing the need for long timesteps.

Contribution

The paper proposes AMP2, a unified, biologically inspired propagation mechanism that enhances SNN performance and scalability across various architectures and data modalities.

Findings

AMP2 improves SNN accuracy with fewer timesteps

It enhances efficiency and reduces latency in SNNs

Effective across MLPs, CNNs, and Transformers

Abstract

Spiking neural networks (SNNs) have demonstrated significant potential in real-time multi-sensor perception tasks due to their event-driven and parameter-efficient characteristics. A key challenge is the timestep-wise iterative update of neuronal hidden states (membrane potentials), which complicates the trade-off between accuracy and latency. SNNs tend to achieve better performance with longer timesteps, inevitably resulting in higher computational overhead and latency compared to artificial neural networks (ANNs). Moreover, many recent advances in SNNs rely on architecture-specific optimizations, which, while effective with fewer timesteps, often limit generalizability and scalability across modalities and models. To address these limitations, we propose Activation-wise Membrane Potential Propagation (AMP2), a unified hidden state update mechanism for SNNs. Inspired by the spatial propagation of membrane potentials in biological neurons, AMP2 enables dynamic transmission of membrane potentials among spatially adjacent neurons, facilitating spatiotemporal integration and cooperative dynamics of hidden states, thereby improving efficiency and accuracy while reducing reliance on extended temporal updates. This simple yet effective strategy significantly enhances SNN performance across various architectures, including MLPs and CNNs for point cloud and event-based data. Furthermore, ablation studies integrating AMP2 into Transformer-based SNNs for classification tasks demonstrate its potential as a general-purpose and efficient solution for spiking neural networks.