Embedded event based object detection with spiking neural network

TL;DR

This paper presents an embedded neuromorphic system using a specialized SNN accelerator for efficient event-based object detection, demonstrating low power consumption and real-world applicability on dedicated hardware.

Contribution

It introduces an embedded neuromorphic testbench with a novel FPGA-based SNN deployment framework for low-power, real-time event-based object detection.

Findings

Achieved 490 mJ per prediction with a 1.08 million parameter SNN.

Successfully deployed and evaluated SNNs on dedicated neuromorphic hardware.

Demonstrated real-world event-based object detection on embedded systems.

Abstract

The complexity of event-based object detection (OD) poses considerable challenges. Spiking Neural Networks (SNNs) show promising results and pave the way for efficient event-based OD. Despite this success, the path to efficient SNNs on embedded devices remains a challenge. This is due to the size of the networks required to accomplish the task and the ability of devices to take advantage of SNNs benefits. Even when "edge" devices are considered, they typically use embedded GPUs that consume tens of watts. In response to these challenges, our research introduces an embedded neuromorphic testbench that utilizes the SPiking Low-power Event-based ArchiTecture (SPLEAT) accelerator. Using an extended version of the Qualia framework, we can train, evaluate, quantize, and deploy spiking neural networks on an FPGA implementation of SPLEAT. We used this testbench to load a state-of-the-art SNN solution, estimate the performance loss associated with deploying the network on dedicated hardware, and run real-world event-based OD on neuromorphic hardware specifically designed for low-power spiking neural networks. Remarkably, our embedded spiking solution, which includes a model with 1.08 million parameters, operates efficiently with 490 mJ per prediction.