Relative Representations of Latent Spaces enable Efficient Semantic Channel Equalization

TL;DR

This paper introduces a semantic equalization algorithm based on relative representations that enables efficient, retraining-free communication between multi-agent systems with different neural network models and datasets.

Contribution

The paper proposes a novel relative representation framework and anchor selection strategy for semantic channel equalization across diverse agents without retraining.

Findings

Effective communication between agents with different models.

Reduced information exchange through anchor-based compression.

Seamless interaction across different neural architectures and datasets.

Abstract

In multi-user semantic communication, language mismatche poses a significant challenge when independently trained agents interact. We present a novel semantic equalization algorithm that enables communication between agents with different languages without additional retraining. Our algorithm is based on relative representations, a framework that enables different agents employing different neural network models to have unified representation. It proceeds by projecting the latent vectors of different models into a common space defined relative to a set of data samples called \textit{anchors}, whose number equals the dimension of the resulting space. A communication between different agents translates to a communication of semantic symbols sampled from this relative space. This approach, in addition to aligning the semantic representations of different agents, allows compressing the amount of information being exchanged, by appropriately selecting the number of anchors. Eventually, we introduce a novel anchor selection strategy, which advantageously determines prototypical anchors, capturing the most relevant information for the downstream task. Our numerical results show the effectiveness of the proposed approach allowing seamless communication between agents with radically different models, including differences in terms of neural network architecture and datasets used for initial training.