A Survey of Multi-sensor Fusion Perception for Embodied AI: Background, Methods, Challenges and Prospects

TL;DR

This survey comprehensively reviews multi-sensor fusion perception (MSFP) in embodied AI, covering various methods, perspectives, and challenges to guide future research across multiple tasks and applications.

Contribution

It offers a task-agnostic, multi-perspective overview of MSFP methods, including multi-modal, multi-view, time-series, and multimodal LLM fusion, addressing limitations of prior single-task surveys.

Findings

Diverse MSFP methods are crucial for embodied AI applications.

Multi-view and time-series fusion are key technical approaches.

Open challenges include integration complexity and real-world deployment.

Abstract

Multi-sensor fusion perception (MSFP) is a key technology for embodied AI, which can serve a variety of downstream tasks (e.g., 3D object detection and semantic segmentation) and application scenarios (e.g., autonomous driving and swarm robotics). Recently, impressive achievements on AI-based MSFP methods have been reviewed in relevant surveys. However, we observe that the existing surveys have some limitations after a rigorous and detailed investigation. For one thing, most surveys are oriented to a single task or research field, such as 3D object detection or autonomous driving. Therefore, researchers in other related tasks often find it difficult to benefit directly. For another, most surveys only introduce MSFP from a single perspective of multi-modal fusion, while lacking consideration of the diversity of MSFP methods, such as multi-view fusion and time-series fusion. To this end, in this paper, we hope to organize MSFP research from a task-agnostic perspective, where methods are reported from various technical views. Specifically, we first introduce the background of MSFP. Next, we review multi-modal and multi-agent fusion methods. A step further, time-series fusion methods are analyzed. In the era of LLM, we also investigate multimodal LLM fusion methods. Finally, we discuss open challenges and future directions for MSFP. We hope this survey can help researchers understand the important progress in MSFP and provide possible insights for future research.