Disentangling Learnable and Memorizable Data via Contrastive Learning for Semantic Communications

TL;DR

This paper introduces a contrastive learning framework that disentangles semantic-rich data from memorizable data, enabling more efficient and human-like semantic communication in future wireless networks.

Contribution

It proposes a novel contrastive learning method to pre-process data, forming high-confidence semantic clusters for efficient semantic communication systems.

Findings

Semantic representation length reduced by 57.22%

Contrastive learning improves semantic impact and minimalism

High-confidence clusters are used as semantic-rich data

Abstract

Achieving artificially intelligent-native wireless networks is necessary for the operation of future 6G applications such as the metaverse. Nonetheless, current communication schemes are, at heart, a mere reconstruction process that lacks reasoning. One key solution that enables evolving wireless communication to a human-like conversation is semantic communications. In this paper, a novel machine reasoning framework is proposed to pre-process and disentangle source data so as to make it semantic-ready. In particular, a novel contrastive learning framework is proposed, whereby instance and cluster discrimination are performed on the data. These two tasks enable increasing the cohesiveness between data points mapping to semantically similar content elements and disentangling data points of semantically different content elements. Subsequently, the semantic deep clusters formed are ranked according to their level of confidence. Deep semantic clusters of highest confidence are considered learnable, semantic-rich data, i.e., data that can be used to build a language in a semantic communications system. The least confident ones are considered, random, semantic-poor, and memorizable data that must be transmitted classically. Our simulation results showcase the superiority of our contrastive learning approach in terms of semantic impact and minimalism. In fact, the length of the semantic representation achieved is minimized by 57.22% compared to vanilla semantic communication systems, thus achieving minimalist semantic representations.