Towards a Foundation Model for Communication Systems

TL;DR

This paper proposes a transformer-based foundation model for communication systems that can handle multi-modal data and estimate various communication features, marking a step toward general AI models in communication.

Contribution

It introduces a novel multi-modal transformer model tailored for communication data, addressing key challenges like tokenization and normalization, and demonstrates its ability to estimate multiple communication features.

Findings

Successfully estimates transmission rank, precoder, Doppler spread, and delay profile.

Addresses key challenges in applying transformers to communication data.

Paves the way for general AI models in communication systems.

Abstract

Artificial Intelligence (AI) has demonstrated unprecedented performance across various domains, and its application to communication systems is an active area of research. While current methods focus on task-specific solutions, the broader trend in AI is shifting toward large general models capable of supporting multiple applications. In this work, we take a step toward a foundation model for communication data--a transformer-based, multi-modal model designed to operate directly on communication data. We propose methodologies to address key challenges, including tokenization, positional embedding, multimodality, variable feature sizes, and normalization. Furthermore, we empirically demonstrate that such a model can successfully estimate multiple features, including transmission rank, selected precoder, Doppler spread, and delay profile.