MmWave Radar and Vision Fusion for Object Detection in Autonomous Driving: A Review

TL;DR

This review paper comprehensively surveys mmWave radar and vision fusion techniques for object detection in autonomous driving, covering methods, datasets, and future directions for multimodal sensor integration.

Contribution

It provides a detailed classification and analysis of sensor deployment, calibration, and fusion methods, including 3D detection and multimodal fusion, highlighting recent advances and future trends.

Findings

Classifies fusion methods into data, decision, and feature levels.

Summarizes datasets and evaluation criteria for autonomous driving detection.

Discusses future prospects of multimodal sensor fusion.

Abstract

With autonomous driving developing in a booming stage, accurate object detection in complex scenarios attract wide attention to ensure the safety of autonomous driving. Millimeter wave (mmWave) radar and vision fusion is a mainstream solution for accurate obstacle detection. This article presents a detailed survey on mmWave radar and vision fusion based obstacle detection methods. First, we introduce the tasks, evaluation criteria, and datasets of object detection for autonomous driving. The process of mmWave radar and vision fusion is then divided into three parts: sensor deployment, sensor calibration, and sensor fusion, which are reviewed comprehensively. Specifically, we classify the fusion methods into data level, decision level, and feature level fusion methods. In addition, we introduce three-dimensional(3D) object detection, the fusion of lidar and vision in autonomous driving and multimodal information fusion, which are promising for the future. Finally, we summarize this article.