RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection

TL;DR

RobuRCDet enhances 3D object detection robustness by integrating a 3D Gaussian Expansion module and a weather-adaptive fusion mechanism to effectively handle sensor noise and environmental disturbances.

Contribution

The paper introduces RobuRCDet, a novel radar-camera fusion model with a 3D Gaussian Expansion module and adaptive fusion, improving robustness under various noise and weather conditions.

Findings

RobuRCDet outperforms existing methods on nuScenes benchmark.

The 3D Gaussian Expansion module reduces radar point inaccuracies.

Adaptive fusion improves detection stability in adverse weather.

Abstract

While recent low-cost radar-camera approaches have shown promising results in multi-modal 3D object detection, both sensors face challenges from environmental and intrinsic disturbances. Poor lighting or adverse weather conditions degrade camera performance, while radar suffers from noise and positional ambiguity. Achieving robust radar-camera 3D object detection requires consistent performance across varying conditions, a topic that has not yet been fully explored. In this work, we first conduct a systematic analysis of robustness in radar-camera detection on five kinds of noises and propose RobuRCDet, a robust object detection model in BEV. Specifically, we design a 3D Gaussian Expansion (3DGE) module to mitigate inaccuracies in radar points, including position, Radar Cross-Section (RCS), and velocity. The 3DGE uses RCS and velocity priors to generate a deformable kernel map and variance for kernel size adjustment and value distribution. Additionally, we introduce a weather-adaptive fusion module, which adaptively fuses radar and camera features based on camera signal confidence. Extensive experiments on the popular benchmark, nuScenes, show that our model achieves competitive results in regular and noisy conditions.