Language-Oriented Semantic Latent Representation for Image Transmission

TL;DR

This paper introduces a language-oriented semantic communication framework that transmits compressed image embeddings along with text, enabling high-quality image reconstruction with minimal data transmission in noisy channels.

Contribution

It proposes a novel semantic communication method combining text and image embeddings with latent diffusion models for efficient image transmission.

Findings

Transmits only 2.09% of original image size.

Achieves higher perceptual similarity than text-only methods.

Effective in noisy communication channels.

Abstract

In the new paradigm of semantic communication (SC), the focus is on delivering meanings behind bits by extracting semantic information from raw data. Recent advances in data-to-text models facilitate language-oriented SC, particularly for text-transformed image communication via image-to-text (I2T) encoding and text-to-image (T2I) decoding. However, although semantically aligned, the text is too coarse to precisely capture sophisticated visual features such as spatial locations, color, and texture, incurring a significant perceptual difference between intended and reconstructed images. To address this limitation, in this paper, we propose a novel language-oriented SC framework that communicates both text and a compressed image embedding and combines them using a latent diffusion model to reconstruct the intended image. Experimental results validate the potential of our approach, which transmits only 2.09\% of the original image size while achieving higher perceptual similarities in noisy communication channels compared to a baseline SC method that communicates only through text.The code is available at https://github.com/ispamm/Img2Img-SC/ .