Motion Robust High-Speed Light-Weighted Object Detection With Event Camera

TL;DR

This paper introduces a novel high-speed, motion-robust object detection pipeline for event cameras, utilizing a new event representation, a temporal encoding module, and a lightweight detector, demonstrating competitive accuracy and efficiency.

Contribution

It presents a new event stream representation (TAF), a bifurcated folding module (BFM), and a lightweight detector (AED) for improved motion robustness and speed in event-based object detection.

Findings

Competitive accuracy on real-scene datasets

Efficient detection with fewer parameters

Robustness across different object velocities

Abstract

In this work, we propose a motion robust and high-speed detection pipeline which better leverages the event data. First, we design an event stream representation called temporal active focus (TAF), which efficiently utilizes the spatial-temporal asynchronous event stream, constructing event tensors robust to object motions. Then, we propose a module called the bifurcated folding module (BFM), which encodes the rich temporal information in the TAF tensor at the input layer of the detector. Following this, we design a high-speed lightweight detector called agile event detector (AED) plus a simple but effective data augmentation method, to enhance the detection accuracy and reduce the model's parameter. Experiments on two typical real-scene event camera object detection datasets show that our method is competitive in terms of accuracy, efficiency, and the number of parameters. By classifying objects into multiple motion levels based on the optical flow density metric, we further illustrated the robustness of our method for objects with different velocities relative to the camera. The codes and trained models are available at https://github.com/HarmoniaLeo/FRLW-EvD .