Scalability and Optimization Strategies for GPU Enhanced Neural Networks (GeNN)

TL;DR

This paper discusses strategies for scaling and optimizing GPU-based neural network simulations, focusing on synaptic weight scaling, sparse representations, and GPU-specific optimizations to improve performance and learning.

Contribution

It introduces methods for scaling synaptic weights and GPU-specific optimization techniques to enhance GeNN's neural network simulation performance.

Findings

Effective synaptic weight scaling improves neural activity.

Sparse synapse representation enhances GPU efficiency.

GPU optimization strategies increase simulation speed.

Abstract

Simulation of spiking neural networks has been traditionally done on high-performance supercomputers or large-scale clusters. Utilizing the parallel nature of neural network computation algorithms, GeNN (GPU Enhanced Neural Network) provides a simulation environment that performs on General Purpose NVIDIA GPUs with a code generation based approach. GeNN allows the users to design and simulate neural networks by specifying the populations of neurons at different stages, their synapse connection densities and the model of individual neurons. In this report we describe work on how to scale synaptic weights based on the configuration of the user-defined network to ensure sufficient spiking and subsequent effective learning. We also discuss optimization strategies particular to GPU computing: sparse representation of synapse connections and occupancy based block-size determination.