Sequential Semantic Generative Communication for Progressive Text-to-Image Generation

TL;DR

This paper introduces a novel communication framework that uses multi-modal generative models to transmit images as text prompts, enabling efficient, progressive image reconstruction through sequential word transmission.

Contribution

It proposes a new system that converts images to text and transmits words sequentially based on information priority, enhancing communication efficiency with generative models.

Findings

Sequential word transmission improves communication efficiency.

The system effectively reconstructs images from text prompts.

Utilizes multi-modal generative models for real-world communication applications.

Abstract

This paper proposes new framework of communication system leveraging promising generation capabilities of multi-modal generative models. Regarding nowadays smart applications, successful communication can be made by conveying the perceptual meaning, which we set as text prompt. Text serves as a suitable semantic representation of image data as it has evolved to instruct an image or generate image through multi-modal techniques, by being interpreted in a manner similar to human cognition. Utilizing text can also reduce the overload compared to transmitting the intact data itself. The transmitter converts objective image to text through multi-model generation process and the receiver reconstructs the image using reverse process. Each word in the text sentence has each syntactic role, responsible for particular piece of information the text contains. For further efficiency in communication load, the transmitter sequentially sends words in priority of carrying the most information until reaches successful communication. Therefore, our primary focus is on the promising design of a communication system based on image-to-text transformation and the proposed schemes for sequentially transmitting word tokens. Our work is expected to pave a new road of utilizing state-of-the-art generative models to real communication systems