MoRAL: Motion-aware Multi-Frame 4D Radar and LiDAR Fusion for Robust 3D Object Detection

TL;DR

MoRAL introduces a motion-aware fusion framework that effectively compensates for inter-frame misalignment and leverages dynamic information from 4D radar and LiDAR data, significantly improving 3D object detection in autonomous driving.

Contribution

The paper presents a novel motion-aware radar encoder and a fusion module that jointly enhance multi-frame radar and LiDAR fusion for robust 3D detection.

Findings

MoRAL achieves the highest mAP of 73.30% in the entire area.

It attains 88.68% mAP in the driving corridor.

It reaches 69.67% AP for pedestrians and 96.25% for cyclists.

Abstract

Reliable autonomous driving systems require accurate detection of traffic participants. To this end, multi-modal fusion has emerged as an effective strategy. In particular, 4D radar and LiDAR fusion methods based on multi-frame radar point clouds have demonstrated the effectiveness in bridging the point density gap. However, they often neglect radar point clouds' inter-frame misalignment caused by object movement during accumulation and do not fully exploit the object dynamic information from 4D radar. In this paper, we propose MoRAL, a motion-aware multi-frame 4D radar and LiDAR fusion framework for robust 3D object detection. First, a Motion-aware Radar Encoder (MRE) is designed to compensate for inter-frame radar misalignment from moving objects. Later, a Motion Attention Gated Fusion (MAGF) module integrate radar motion features to guide LiDAR features to focus on dynamic foreground objects. Extensive evaluations on the View-of-Delft (VoD) dataset demonstrate that MoRAL outperforms existing methods, achieving the highest mAP of 73.30% in the entire area and 88.68% in the driving corridor. Notably, our method also achieves the best AP of 69.67% for pedestrians in the entire area and 96.25% for cyclists in the driving corridor.