CrossFusion: Interleaving Cross-modal Complementation for Noise-resistant 3D Object Detection

TL;DR

CrossFusion introduces a robust cross-modal complementation approach for 3D object detection that effectively leverages both LiDAR and camera data, improving accuracy and noise resistance without extra depth estimation networks.

Contribution

The paper proposes CrossFusion, a novel scheme that enhances multi-modal fusion by full utilization of camera and LiDAR features, increasing noise robustness and outperforming existing methods.

Findings

Outperforms state-of-the-art methods without extra depth networks

Increases 5.2% mAP and 2.4% NDS under noisy conditions

Demonstrates robustness without re-training for specific malfunctions

Abstract

The combination of LiDAR and camera modalities is proven to be necessary and typical for 3D object detection according to recent studies. Existing fusion strategies tend to overly rely on the LiDAR modal in essence, which exploits the abundant semantics from the camera sensor insufficiently. However, existing methods cannot rely on information from other modalities because the corruption of LiDAR features results in a large domain gap. Following this, we propose CrossFusion, a more robust and noise-resistant scheme that makes full use of the camera and LiDAR features with the designed cross-modal complementation strategy. Extensive experiments we conducted show that our method not only outperforms the state-of-the-art methods under the setting without introducing an extra depth estimation network but also demonstrates our model's noise resistance without re-training for the specific malfunction scenarios by increasing 5.2\% mAP and 2.4\% NDS.