Sparse Convolutional Recurrent Learning for Efficient Event-based Neuromorphic Object Detection

TL;DR

This paper introduces SEED, a sparse convolutional recurrent network that significantly improves the efficiency of event-based object detection on neuromorphic hardware, enabling real-time, low-power applications.

Contribution

The paper proposes sparse convolutional recurrent learning and the SEED architecture, achieving high activation sparsity and improved efficiency for event-based object detection.

Findings

Achieves over 92% activation sparsity in recurrent units.

Sets new benchmarks in computational efficiency for event detection.

Demonstrates energy-efficient, low-latency processing on neuromorphic hardware.

Abstract

Leveraging the high temporal resolution and dynamic range, object detection with event cameras can enhance the performance and safety of automotive and robotics applications in real-world scenarios. However, processing sparse event data requires compute-intensive convolutional recurrent units, complicating their integration into resource-constrained edge applications. Here, we propose the Sparse Event-based Efficient Detector (SEED) for efficient event-based object detection on neuromorphic processors. We introduce sparse convolutional recurrent learning, which achieves over 92% activation sparsity in recurrent processing, vastly reducing the cost for spatiotemporal reasoning on sparse event data. We validated our method on Prophesee's 1 Mpx and Gen1 event-based object detection datasets. Notably, SEED sets a new benchmark in computational efficiency for event-based object detection which requires long-term temporal learning. Compared to state-of-the-art methods, SEED significantly reduces synaptic operations while delivering higher or same-level mAP. Our hardware simulations showcase the critical role of SEED's hardware-aware design in achieving energy-efficient and low-latency neuromorphic processing.