A Recurrent YOLOv8-based framework for Event-Based Object Detection

TL;DR

This paper presents ReYOLOv8, a novel event-based object detection framework that improves accuracy and efficiency in challenging environments by leveraging spatiotemporal modeling and specialized data augmentation techniques.

Contribution

The study introduces ReYOLOv8, integrating event data encoding and augmentation methods to enhance detection performance and reduce model complexity in real-time applications.

Findings

Outperformed comparable approaches on GEN1 dataset with up to 5% mAP improvement.

Achieved significant model size reduction of up to 14.5x while maintaining accuracy.

Enhanced detection speed, processing between 9.2ms and 15.5ms per frame.

Abstract

Object detection is crucial in various cutting-edge applications, such as autonomous vehicles and advanced robotics systems, primarily relying on data from conventional frame-based RGB sensors. However, these sensors often struggle with issues like motion blur and poor performance in challenging lighting conditions. In response to these challenges, event-based cameras have emerged as an innovative paradigm. These cameras, mimicking the human eye, demonstrate superior performance in environments with fast motion and extreme lighting conditions while consuming less power. This study introduces ReYOLOv8, an advanced object detection framework that enhances a leading frame-based detection system with spatiotemporal modeling capabilities. We implemented a low-latency, memory-efficient method for encoding event data to boost the system's performance. We also developed a novel data augmentation technique tailored to leverage the unique attributes of event data, thus improving detection accuracy. Our models outperformed all comparable approaches in the GEN1 dataset, focusing on automotive applications, achieving mean Average Precision (mAP) improvements of 5%, 2.8%, and 2.5% across nano, small, and medium scales, respectively.These enhancements were achieved while reducing the number of trainable parameters by an average of 4.43% and maintaining real-time processing speeds between 9.2ms and 15.5ms. On the PEDRo dataset, which targets robotics applications, our models showed mAP improvements ranging from 9% to 18%, with 14.5x and 3.8x smaller models and an average speed enhancement of 1.67x.