Joint Semantic-Channel Coding and Modulation for Token Communications

TL;DR

This paper introduces a joint semantic-channel coding and modulation scheme for token communication, specifically for transmitting point cloud tokens efficiently and reliably over channels, outperforming traditional methods.

Contribution

It proposes a novel JSCCM scheme with Transformer-based encoders and adaptive rate allocation for improved token transmission in 3D point cloud data.

Findings

Achieves over 1dB gain in reconstruction quality

More than 6x compression ratio in modulated symbols

Outperforms separate coding methods in simulations

Abstract

In recent years, the Transformer architecture has achieved outstanding performance across a wide range of tasks and modalities. Token is the unified input and output representation in Transformer-based models, which has become a fundamental information unit. In this work, we consider the problem of token communication, studying how to transmit tokens efficiently and reliably. Point cloud, a prevailing three-dimensional format which exhibits a more complex spatial structure compared to image or video, is chosen to be the information source. We utilize the set abstraction method to obtain point tokens. Subsequently, to get a more informative and transmission-friendly representation based on tokens, we propose a joint semantic-channel and modulation (JSCCM) scheme for the token encoder, mapping point tokens to standard digital constellation points (modulated tokens). Specifically, the JSCCM consists of two parallel Point Transformer-based encoders and a differential modulator which combines the Gumel-softmax and soft quantization methods. Besides, the rate allocator and channel adapter are developed, facilitating adaptive generation of high-quality modulated tokens conditioned on both semantic information and channel conditions. Extensive simulations demonstrate that the proposed method outperforms both joint semantic-channel coding and traditional separate coding, achieving over 1dB gain in reconstruction and more than 6x compression ratio in modulated symbols.