Enhancing Event-based Object Detection with Monocular Normal Maps

TL;DR

This paper introduces NRE-Net, a novel trimodal framework that enhances event-based object detection in autonomous driving by integrating surface normal maps, RGB images, and event data, improving accuracy under challenging lighting conditions.

Contribution

The paper presents NRE-Net, a new framework that effectively combines geometric priors, appearance, and event dynamics for improved detection performance.

Findings

Incorporating geometric priors improves AP50 by 3.0% over dual-modal baselines.

NRE-Net outperforms existing fusion methods, e.g., SFNet (+2.7%) and SODFormer (+7.1%).

Extensive evaluations demonstrate robustness under complex illumination conditions.

Abstract

Object detection in autonomous driving is frequently compromised by complex illumination. While event cameras offer a robust solution, they are susceptible to sudden contrast changes such as reflections which often trigger dense, misleading event signals. To overcome this, we leverage RGB-derived surface normal maps as explicit geometric constraints. Crucially, even when RGB degrades, they preserve low-frequency structural priors that effectively assist in event-based detection. Consequently, we present NRE-Net, a trimodal framework that integrates structural priors from surface Normal maps, appearance context from RGB images, and high-frequency dynamics from Events. The Adaptive Dual-stream Fusion Module (ADFM) first aligns geometric and appearance cues, followed by the Event-modality Aware Fusion Module (EAFM) which selectively integrates event dynamics. Extensive evaluations on DSEC-Det-sub and PKU-DAVIS-SOD demonstrate that incorporating geometric priors yields an additional 3.0% AP50 gain over dual-modal baselines, while our approach consistently outperforms fusion methods such as SFNet (+2.7%) and SODFormer (+7.1%).