Multi-GPU SNN Simulation with Static Load Balancing

TL;DR

This paper introduces a scalable multi-GPU spiking neural network simulator that leverages a novel spike transmission algorithm, model parallelism, and static load balancing to efficiently simulate large neural models.

Contribution

The paper presents a new multi-GPU SNN simulator with a cache-aware spike transmission, model parallel distribution, and static load balancing, outperforming existing simulators.

Findings

Faster simulation times compared to state-of-the-art.

Lower memory consumption.

Linear scalability with the number of GPUs.

Abstract

We present a SNN simulator which scales to millions of neurons, billions of synapses, and 8 GPUs. This is made possible by 1) a novel, cache-aware spike transmission algorithm 2) a model parallel multi-GPU distribution scheme and 3) a static, yet very effective load balancing strategy. The simulator further features an easy to use API and the ability to create custom models. We compare the proposed simulator against two state of the art ones on a series of benchmarks using three well-established models. We find that our simulator is faster, consumes less memory, and scales linearly with the number of GPUs.