Integrating Multi-Modal Sensors: A Review of Fusion Techniques for Intelligent Vehicles

TL;DR

This review comprehensively analyzes multi-sensor fusion techniques in autonomous vehicles, emphasizing deep learning methods, datasets, and emerging trends like VLMs and LLMs to improve perception and robustness.

Contribution

It systematically categorizes fusion strategies, reviews deep learning approaches, and discusses future trends and datasets for multi-sensor fusion in autonomous driving.

Findings

Deep learning-based fusion methods improve perception accuracy.

Key datasets enable testing under adverse weather and urban scenarios.

Emerging trends include integration of VLMs and LLMs for enhanced system robustness.

Abstract

Multi-sensor fusion plays a critical role in enhancing perception for autonomous driving, overcoming individual sensor limitations, and enabling comprehensive environmental understanding. This paper first formalizes multi-sensor fusion strategies into data-level, feature-level, and decision-level categories and then provides a systematic review of deep learning-based methods corresponding to each strategy. We present key multi-modal datasets and discuss their applicability in addressing real-world challenges, particularly in adverse weather conditions and complex urban environments. Additionally, we explore emerging trends, including the integration of Vision-Language Models (VLMs), Large Language Models (LLMs), and the role of sensor fusion in end-to-end autonomous driving, highlighting its potential to enhance system adaptability and robustness. Our work offers valuable insights into current methods and future directions for multi-sensor fusion in autonomous driving.