Even Faster SNN Simulation with Lazy+Event-driven Plasticity and Shared Atomics

TL;DR

This paper introduces two innovative optimizations for clock-based SNN simulators, significantly enhancing the speed of spike timing dependent plasticity and spike delivery, leading to faster and more efficient neural network simulations.

Contribution

The paper presents novel lazy+event-driven plasticity and shared memory spike delivery techniques that outperform existing methods in SNN simulation speed.

Findings

Achieved 1.5x-2x speedup in STDP computation.

Realized 2x-2.5x faster spike delivery.

Demonstrated improvements across three models and three simulators.

Abstract

We present two novel optimizations that accelerate clock-based spiking neural network (SNN) simulators. The first one targets spike timing dependent plasticity (STDP). It combines lazy- with event-driven plasticity and efficiently facilitates the computation of pre- and post-synaptic spikes using bitfields and integer intrinsics. It offers higher bandwidth than event-driven plasticity alone and achieves a 1.5x-2x speedup over our closest competitor. The second optimization targets spike delivery. We partition our graph representation in a way that bounds the number of neurons that need be updated at any given time which allows us to perform said update in shared memory instead of global memory. This is 2x-2.5x faster than our closest competitor. Both optimizations represent the final evolutionary stages of years of iteration on STDP and spike delivery inside "Spice" (/spaIk/), our state of the art SNN simulator. The proposed optimizations are not exclusive to our graph representation or pipeline but are applicable to a multitude of simulator designs. We evaluate our performance on three well-established models and compare ourselves against three other state of the art simulators.