A Multi-Modal Foundational Model for Wireless Communication and Sensing

TL;DR

This paper presents a versatile multi-modal foundational AI model for wireless systems that generalizes across tasks and environments, reducing retraining needs and improving robustness in physical-layer applications.

Contribution

It introduces a physics-aware, task-agnostic pretraining framework with a physical token, enabling transferability and robustness in wireless communication and sensing tasks.

Findings

Superior generalization across diverse tasks

Robustness to deployment shifts

Reduced data requirements for downstream tasks

Abstract

Artificial intelligence is a key enabler for next-generation wireless communication and sensing. Yet, today's learning-based wireless techniques do not generalize well: most models are task-specific, environment-dependent, and limited to narrow sensing modalities, requiring costly retraining when deployed in new scenarios. This work introduces a task-agnostic, multi-modal foundational model for physical-layer wireless systems that learns transferable, physics-aware representations across heterogeneous modalities, enabling robust generalization across tasks and environments. Our framework employs a physics-guided self-supervised pretraining strategy incorporating a dedicated physical token to capture cross-modal physical correspondences governed by electromagnetic propagation. The learned representations enable efficient adaptation to diverse downstream tasks, including massive multi-antenna optimization, wireless channel estimation, and device localization, using limited labeled data. Our extensive evaluations demonstrate superior generalization, robustness to deployment shifts, and reduced data requirements compared to task-specific baselines.