RaLiBEV: Radar and LiDAR BEV Fusion Learning for Anchor Box Free Object Detection Systems

TL;DR

This paper introduces RaLiBEV, a novel radar and LiDAR fusion system for object detection in autonomous driving, improving accuracy and robustness in adverse weather by using an anchor box-free approach and transformer modules.

Contribution

The paper presents a new BEV fusion learning method that combines radar and LiDAR data without anchor boxes, employing innovative label strategies and a transformer module for enhanced detection accuracy.

Findings

Outperforms state-of-the-art by 13.1% and 19.0% AP at IoU 0.8 in foggy conditions.

Demonstrates robustness of radar-LiDAR fusion in adverse weather.

Uses Oxford Radar RobotCar dataset for validation.

Abstract

In autonomous driving, LiDAR and radar are crucial for environmental perception. LiDAR offers precise 3D spatial sensing information but struggles in adverse weather like fog. Conversely, radar signals can penetrate rain or mist due to their specific wavelength but are prone to noise disturbances. Recent state-of-the-art works reveal that the fusion of radar and LiDAR can lead to robust detection in adverse weather. The existing works adopt convolutional neural network architecture to extract features from each sensor data, then align and aggregate the two branch features to predict object detection results. However, these methods have low accuracy of predicted bounding boxes due to a simple design of label assignment and fusion strategies. In this paper, we propose a bird's-eye view fusion learning-based anchor box-free object detection system, which fuses the feature derived from the radar range-azimuth heatmap and the LiDAR point cloud to estimate possible objects. Different label assignment strategies have been designed to facilitate the consistency between the classification of foreground or background anchor points and the corresponding bounding box regressions. Furthermore, the performance of the proposed object detector is further enhanced by employing a novel interactive transformer module. The superior performance of the methods proposed in this paper has been demonstrated using the recently published Oxford Radar RobotCar dataset. Our system's average precision significantly outperforms the state-of-the-art method by 13.1% and 19.0% at Intersection of Union (IoU) of 0.8 under 'Clear+Foggy' training conditions for 'Clear' and 'Foggy' testing, respectively.