Integrating Pre-Trained Language Model with Physical Layer Communications

TL;DR

This paper presents a practical on-device AI communication framework that integrates pre-trained language models with physical layer functions, achieving efficient, noise-robust, and generalizable wireless communication.

Contribution

It introduces a novel framework combining end-to-end training, VQ-VAE, and pre-trained transformers for improved wireless communication performance.

Findings

50% reduction in transmission size

Enhanced noise robustness under 3GPP models

Strong generalization across scenarios

Abstract

The burgeoning field of on-device AI communication, where devices exchange information directly through embedded foundation models, such as language models (LMs), requires robust, efficient, and generalizable communication frameworks. However, integrating these frameworks with existing wireless systems and effectively managing noise and bit errors pose significant challenges. In this work, we introduce a practical ondevice AI communication framework, integrated with physical layer (PHY) communication functions, demonstrated through its performance on a link-level simulator. Our framework incorporates end-to-end training with channel noise to enhance resilience, incorporates vector quantized variational autoencoders (VQ-VAE) for efficient and robust communication, and utilizes pre-trained encoder-decoder transformers for improved generalization capabilities. Simulations, across various communication scenarios, reveal that our framework achieves a 50% reduction in transmission size while demonstrating substantial generalization ability and noise robustness under standardized 3GPP channel models.