Adaptive Semantic Token Selection for AI-native Goal-oriented Communications

TL;DR

This paper introduces a dynamic, trainable semantic token selection mechanism for transformer-based AI-native goal-oriented communications, improving adaptability to variable bandwidth and latency while enhancing interpretability.

Contribution

It presents a novel, trainable token selection method that adapts to input and network constraints, outperforming existing mechanisms and enabling interpretable AI-native communication models.

Findings

Improves token selection accuracy across diverse constraints

Reduces need for multiple specialized architectures

Enhances interpretability of communication models

Abstract

In this paper, we propose a novel design for AI-native goal-oriented communications, exploiting transformer neural networks under dynamic inference constraints on bandwidth and computation. Transformers have become the standard architecture for pretraining large-scale vision and text models, and preliminary results have shown promising performance also in deep joint source-channel coding (JSCC). Here, we consider a dynamic model where communication happens over a channel with variable latency and bandwidth constraints. Leveraging recent works on conditional computation, we exploit the structure of the transformer blocks and the multihead attention operator to design a trainable semantic token selection mechanism that learns to select relevant tokens (e.g., image patches) from the input signal. This is done dynamically, on a per-input basis, with a rate that can be chosen as an additional input by the user. We show that our model improves over state-of-the-art token selection mechanisms, exhibiting high accuracy for a wide range of latency and bandwidth constraints, without the need for deploying multiple architectures tailored to each constraint. Last, but not least, the proposed token selection mechanism helps extract powerful semantics that are easy to understand and explain, paving the way for interpretable-by-design models for the next generation of AI-native communication systems.