LiRaFusion: Deep Adaptive LiDAR-Radar Fusion for 3D Object Detection

TL;DR

LiRaFusion introduces a deep learning framework that adaptively fuses LiDAR and radar data for enhanced 3D object detection, significantly outperforming existing methods on the nuScenes dataset.

Contribution

The paper presents novel early and middle fusion modules with a gated network for adaptive feature fusion in LiDAR-radar 3D detection.

Findings

Achieves notable performance improvements on nuScenes dataset.

Effectively leverages complementary LiDAR and radar information.

Demonstrates the effectiveness of the proposed fusion modules.

Abstract

We propose LiRaFusion to tackle LiDAR-radar fusion for 3D object detection to fill the performance gap of existing LiDAR-radar detectors. To improve the feature extraction capabilities from these two modalities, we design an early fusion module for joint voxel feature encoding, and a middle fusion module to adaptively fuse feature maps via a gated network. We perform extensive evaluation on nuScenes to demonstrate that LiRaFusion leverages the complementary information of LiDAR and radar effectively and achieves notable improvement over existing methods.