Context-Aware Iterative Token Detection and Masked Transmission for Wireless Token Communication

TL;DR

This paper introduces a context-aware token communication framework for wireless channels, leveraging pretrained language models to improve transmission efficiency and sentence reconstruction quality through iterative detection and adaptive masking.

Contribution

It presents a novel framework combining MLM-guided context modeling with iterative detection and masking strategies for wireless token communication.

Findings

Significant improvement in sentence reconstruction quality.

Effective rate adaptation under various channel conditions.

Reduced transmission rate through context-aware masking.

Abstract

The success of large-scale language models has established tokens as compact and meaningful units for natural-language representation, which motivates token communication over wireless channels, where tokens are considered fundamental units for wireless transmission. We propose a context-aware token communication framework that uses a pretrained masked language model (MLM) as a shared contextual probability model between the transmitter (Tx) and receiver (Rx). At Rx, we develop an iterative token detection method that jointly exploits MLM-guided contextual priors and channel observations based on a Bayesian perspective. At Tx, we additionally introduce a context-aware masking strategy which skips highly predictable token transmission to reduce transmission rate. Simulation results demonstrate that the proposed framework substantially improves reconstructed sentence quality and supports effective rate adaptation under various channel conditions.