MutualForce: Mutual-Aware Enhancement for 4D Radar-LiDAR 3D Object Detection

TL;DR

This paper introduces MutualForce, a novel framework that mutually enhances radar and LiDAR features for 3D object detection, significantly improving accuracy by addressing modality misalignment and information loss.

Contribution

The paper proposes a mutual-aware enhancement method for radar and LiDAR data, leveraging indicative features and shape information to improve 3D detection performance.

Findings

Achieves a 71.76% mAP on VoD dataset

Improves car AP by over 4%

Outperforms existing fusion methods

Abstract

Radar and LiDAR have been widely used in autonomous driving as LiDAR provides rich structure information, and radar demonstrates high robustness under adverse weather. Recent studies highlight the effectiveness of fusing radar and LiDAR point clouds. However, challenges remain due to the modality misalignment and information loss during feature extractions. To address these issues, we propose a 4D radar-LiDAR framework to mutually enhance their representations. Initially, the indicative features from radar are utilized to guide both radar and LiDAR geometric feature learning. Subsequently, to mitigate their sparsity gap, the shape information from LiDAR is used to enrich radar BEV features. Extensive experiments on the View-of-Delft (VoD) dataset demonstrate our approach's superiority over existing methods, achieving the highest mAP of 71.76% across the entire area and 86.36\% within the driving corridor. Especially for cars, we improve the AP by 4.17% and 4.20% due to the strong indicative features and symmetric shapes.