Cross-Domain Spatial Matching for Camera and Radar Sensor Data Fusion in Autonomous Vehicle Perception System

TL;DR

This paper introduces a novel cross-domain spatial matching method for fusing camera and radar data to enhance 3D object detection in autonomous vehicles, demonstrating superior performance on the NuScenes dataset.

Contribution

It presents a new cross-domain spatial matching technique that effectively combines camera and radar data for improved 3D object detection in autonomous systems.

Findings

Outperforms single-sensor methods in 3D detection accuracy

Achieves competitive results with state-of-the-art fusion techniques

Validated on the NuScenes dataset

Abstract

In this paper, we propose a novel approach to address the problem of camera and radar sensor fusion for 3D object detection in autonomous vehicle perception systems. Our approach builds on recent advances in deep learning and leverages the strengths of both sensors to improve object detection performance. Precisely, we extract 2D features from camera images using a state-of-the-art deep learning architecture and then apply a novel Cross-Domain Spatial Matching (CDSM) transformation method to convert these features into 3D space. We then fuse them with extracted radar data using a complementary fusion strategy to produce a final 3D object representation. To demonstrate the effectiveness of our approach, we evaluate it on the NuScenes dataset. We compare our approach to both single-sensor performance and current state-of-the-art fusion methods. Our results show that the proposed approach achieves superior performance over single-sensor solutions and could directly compete with other top-level fusion methods.