Real Time Scheduling Framework for Multi Object Detection via Spiking Neural Networks

TL;DR

This paper introduces RT-SNN, a real-time scheduling framework for multi-object detection using spiking neural networks on autonomous mobile agents, balancing timing guarantees and high accuracy with energy efficiency.

Contribution

It presents the first system design to achieve real-time performance and high accuracy in SNN-based multi-object detection for autonomous mobile agents.

Findings

RT-SNN effectively balances timing and accuracy requirements.

The framework improves energy efficiency in SNN-based detection.

Experimental results validate the approach on Spiking-YOLO.

Abstract

Given the energy constraints in autonomous mobile agents (AMAs), such as unmanned vehicles, spiking neural networks (SNNs) are increasingly favored as a more efficient alternative to traditional artificial neural networks. AMAs employ multi-object detection (MOD) from multiple cameras to identify nearby objects while ensuring two essential objectives, (R1) timing guarantee and (R2) high accuracy for safety. In this paper, we propose RT-SNN, the first system design, aiming at achieving R1 and R2 in SNN-based MOD systems on AMAs. Leveraging the characteristic that SNNs gather feature data of input image termed as membrane potential, through iterative computation over multiple timesteps, RT-SNN provides multiple execution options with adjustable timesteps and a novel method for reusing membrane potential to support R1. Then, it captures how these execution strategies influence R2 by introducing a novel notion of mean absolute error and membrane confidence. Further, RT-SNN develops a new scheduling framework consisting of offline schedulability analysis for R1 and a run-time scheduling algorithm for R2 using the notion of membrane confidence. We deployed RT-SNN to Spiking-YOLO, the SNN-based MOD model derived from ANN-to-SNN conversion, and our experimental evaluation confirms its effectiveness in meeting the R1 and R2 requirements while providing significant energy efficiency.