Benchmarking neuromorphic systems with Nengo

TL;DR

Nengo is a flexible tool for simulating neural models on various platforms, and its test suite can benchmark neuromorphic hardware performance.

Contribution

Nengo's test suite is proposed as a benchmark for neuromorphic hardware, enabling unbiased comparisons of performance and accuracy.

Findings

Nengo's test suite can be used to benchmark neuromorphic hardware across multiple backends.

Four benchmark models were implemented to compare performance and accuracy across five backends.

Some backends were found to perform more accurately or quickly in specific scenarios.

Abstract

Nengo is a software package for designing and simulating large-scale neural models. Nengo is architected such that the same Nengo model can be simulated on any of several Nengo backends with few to no modifications. Backends translate a model to specific platforms, which include GPUs and neuromorphic hardware. Nengo also contains a large test suite that can be run with any backend and focuses primarily on functional performance. We propose that Nengo's large test suite can be used to benchmark neuromorphic hardware's functional performance and simulation speed in an efficient, unbiased, and future-proof manner. We implement four benchmark models and show that Nengo can collect metrics across five different backends that identify situations in which some backends perform more accurately or quickly.