WD-DETR: Wavelet Denoising-Enhanced Real-Time Object Detection Transformer for Robot Perception with Event Cameras

TL;DR

WD-DETR introduces a wavelet denoising-enhanced transformer architecture for real-time object detection using event cameras, effectively reducing noise, improving detection accuracy, and achieving high frame rates suitable for robotic perception.

Contribution

The paper proposes a novel wavelet denoising method integrated into a transformer-based network for event camera data, enhancing real-time detection performance and robustness.

Findings

Outperforms state-of-the-art methods on three datasets.

Achieves approximately 35 FPS on NVIDIA Jetson Orin NX.

Effectively reduces noise in dense event representations.

Abstract

Previous studies on event camera sensing have demonstrated certain detection performance using dense event representations. However, the accumulated noise in such dense representations has received insufficient attention, which degrades the representation quality and increases the likelihood of missed detections. To address this challenge, we propose the Wavelet Denoising-enhanced DEtection TRansformer, i.e., WD-DETR network, for event cameras. In particular, a dense event representation is presented first, which enables real-time reconstruction of events as tensors. Then, a wavelet transform method is designed to filter noise in the event representations. Such a method is integrated into the backbone for feature extraction. The extracted features are subsequently fed into a transformer-based network for object prediction. To further reduce inference time, we incorporate the Dynamic Reorganization Convolution Block (DRCB) as a fusion module within the hybrid encoder. The proposed method has been evaluated on three event-based object detection datasets, i.e., DSEC, Gen1, and 1Mpx. The results demonstrate that WD-DETR outperforms tested state-of-the-art methods. Additionally, we implement our approach on a common onboard computer for robots, the NVIDIA Jetson Orin NX, achieving a high frame rate of approximately 35 FPS using TensorRT FP16, which is exceptionally well-suited for real-time perception of onboard robotic systems.