Wavelet-based Multi-View Fusion of 4D Radar Tensor and Camera for Robust 3D Object Detection

TL;DR

WRCFormer is a novel framework that fuses raw 4D radar data with camera images using wavelet-based attention and geometry-guided fusion, achieving robust 3D object detection in adverse weather conditions.

Contribution

It introduces a wavelet attention module and a progressive fusion strategy for efficient multi-view radar and camera data integration, improving detection accuracy and robustness.

Findings

Achieves state-of-the-art performance on K-Radar benchmark.

Surpasses existing models by 2.4% overall accuracy.

Demonstrates robustness in sleet and adverse weather conditions.

Abstract

4D millimeter-wave (mmWave) radar has been widely adopted in autonomous driving and robot perception due to its low cost and all-weather robustness. However, point-cloud-based radar representations suffer from information loss due to multi-stage signal processing, while directly utilizing raw 4D radar tensors incurs prohibitive computational costs. To address these challenges, we propose WRCFormer, a novel 3D object detection framework that efficiently fuses raw 4D radar cubes with camera images via decoupled multi-view radar representations. Our approach introduces two key components: (1) A Wavelet Attention Module embedded in a wavelet-based Feature Pyramid Network (FPN), which enhances the representation of sparse radar signals and image data by capturing joint spatial-frequency features, thereby mitigating information loss while maintaining computational efficiency. (2) A Geometry-guided Progressive Fusion mechanism, a two-stage query-based fusion strategy that progressively aligns multi-view radar and visual features through geometric priors, enabling modality-agnostic and efficient integration without overwhelming computational overhead. Extensive experiments on the K-Radar benchmark show that WRCFormer achieves state-of-the-art performance, surpassing the best existing model by approximately 2.4% in all scenarios and 1.6% in sleet conditions, demonstrating strong robustness in adverse weather.