A Unified Cloud-Edge-Terminal Framework for Multimodal Integrated Sensing and Communication

TL;DR

This paper proposes a comprehensive cloud-edge-terminal framework for multimodal sensing and communication in 6G, integrating heterogeneous sensors and AI technologies to enhance robustness, semantic awareness, and adaptive operation modes.

Contribution

It introduces a novel hierarchical CET framework with three adaptive modes, addressing challenges in multimodal ISAC and enabling scalable, task-oriented perception.

Findings

GFM achieves highest sensing accuracy

PIM minimizes communication latency

CRM balances performance and efficiency

Abstract

The transition to 6G calls for tightly integrated sensing and communication to support mission-critical services such as autonomous driving, embodied AI, and high-precision telemedicine. However, most existing ISAC designs rely on a single sensing modality (often RF), which limits environmental understanding and becomes a bottleneck in complex and dynamic scenes. This motivates a shift from single-modal to multimodal ISAC, where heterogeneous sensors (e.g., radar, LiDAR, and cameras) complement each other to improve robustness and semantic awareness. In this article, we first summarize key challenges for multimodal ISAC, including heterogeneous fusion, communication overhead, and scalable system design. We then highlight three enabling technologies: large AI models, semantic communications, and multi-agent systems, and discuss how their combination can enable task-oriented multimodal perception. Building on these insights, we propose a unified cloud-edge-terminal (CET) framework that hierarchically distributes intelligence and supports three adaptive operation modes: global fusion mode (GFM), cooperative relay mode (CRM), and peer interaction mode (PIM). A case study evaluates the framework across three modes, demonstrating that GFM achieves the highest accuracy, PIM minimizes latency, and CRM strikes an optimal balance between performance and efficiency. Finally, we conclude with open research issues and future directions.