Spiking Neural Networks for Frame-based and Event-based Single Object Localization

TL;DR

This paper introduces a spiking neural network for single object localization using frame- and event-based sensors, demonstrating improved accuracy, robustness, and energy efficiency over traditional methods, and analyzing the effects of neural coding schemes.

Contribution

It presents a novel surrogate gradient trained spiking neural network approach for object localization, with comprehensive analysis of sensor noise and encoding impacts.

Findings

Competitive or better accuracy than artificial neural networks.

Enhanced robustness against sensor corruptions.

Lower energy consumption compared to baseline models.

Abstract

Spiking neural networks have shown much promise as an energy-efficient alternative to artificial neural networks. However, understanding the impacts of sensor noises and input encodings on the network activity and performance remains difficult with common neuromorphic vision baselines like classification. Therefore, we propose a spiking neural network approach for single object localization trained using surrogate gradient descent, for frame- and event-based sensors. We compare our method with similar artificial neural networks and show that our model has competitive/better performance in accuracy, robustness against various corruptions, and has lower energy consumption. Moreover, we study the impact of neural coding schemes for static images in accuracy, robustness, and energy efficiency. Our observations differ importantly from previous studies on bio-plausible learning rules, which helps in the design of surrogate gradient trained architectures, and offers insight to design priorities in future neuromorphic technologies in terms of noise characteristics and data encoding methods.