Multi-modal Sensor Fusion for Auto Driving Perception: A Survey

TL;DR

This survey reviews multi-modal sensor fusion methods for autonomous driving perception, analyzing over 50 papers, proposing a new taxonomy, and discussing current challenges and future research opportunities.

Contribution

It introduces a novel taxonomy for classifying multi-modal fusion methods in autonomous driving perception, enhancing understanding of existing approaches and guiding future research.

Findings

Proposes a new taxonomy dividing fusion methods into two major and four minor classes.

Analyzes over 50 papers on LiDAR and camera-based perception methods.

Discusses current challenges and potential research directions in multi-modal fusion.

Abstract

Multi-modal fusion is a fundamental task for the perception of an autonomous driving system, which has recently intrigued many researchers. However, achieving a rather good performance is not an easy task due to the noisy raw data, underutilized information, and the misalignment of multi-modal sensors. In this paper, we provide a literature review of the existing multi-modal-based methods for perception tasks in autonomous driving. Generally, we make a detailed analysis including over 50 papers leveraging perception sensors including LiDAR and camera trying to solve object detection and semantic segmentation tasks. Different from traditional fusion methodology for categorizing fusion models, we propose an innovative way that divides them into two major classes, four minor classes by a more reasonable taxonomy in the view of the fusion stage. Moreover, we dive deep into the current fusion methods, focusing on the remaining problems and open-up discussions on the potential research opportunities. In conclusion, what we expect to do in this paper is to present a new taxonomy of multi-modal fusion methods for the autonomous driving perception tasks and provoke thoughts of the fusion-based techniques in the future.