ROFusion: Efficient Object Detection using Hybrid Point-wise Radar-Optical Fusion

TL;DR

ROFusion introduces a hybrid point-wise Radar-Optical fusion method that leverages dense contextual information from radar and camera data, significantly improving object detection performance in autonomous driving.

Contribution

The paper presents a novel multi-modal fusion framework with a local coordinate formulation, enhancing radar and optical data integration for better object detection.

Findings

Achieved 97.69% recall in object detection, outperforming state-of-the-art methods.

Demonstrated the effectiveness of dense contextual information from radar and images.

Validated key design choices through extensive ablation studies.

Abstract

Radars, due to their robustness to adverse weather conditions and ability to measure object motions, have served in autonomous driving and intelligent agents for years. However, Radar-based perception suffers from its unintuitive sensing data, which lack of semantic and structural information of scenes. To tackle this problem, camera and Radar sensor fusion has been investigated as a trending strategy with low cost, high reliability and strong maintenance. While most recent works explore how to explore Radar point clouds and images, rich contextual information within Radar observation are discarded. In this paper, we propose a hybrid point-wise Radar-Optical fusion approach for object detection in autonomous driving scenarios. The framework benefits from dense contextual information from both the range-doppler spectrum and images which are integrated to learn a multi-modal feature representation. Furthermore, we propose a novel local coordinate formulation, tackling the object detection task in an object-centric coordinate. Extensive results show that with the information gained from optical images, we could achieve leading performance in object detection (97.69\% recall) compared to recent state-of-the-art methods FFT-RadNet (82.86\% recall). Ablation studies verify the key design choices and practicability of our approach given machine generated imperfect detections. The code will be available at https://github.com/LiuLiu-55/ROFusion.