Visual Fidelity Index for Generative Semantic Communications with Critical Information Embedding

TL;DR

This paper introduces a hybrid generative semantic communication system with critical information embedding and a new visual fidelity metric, improving image reconstruction quality and system adaptability in 6G networks.

Contribution

It proposes a novel semantic filtering approach for critical feature extraction and a new GVIF metric for visual quality evaluation, enhancing Gen-SemCom performance.

Findings

GVIF correlates well with PSNR and FID scores.

The system achieves higher PSNR and lower FID than benchmarks.

Adaptive control of features improves visual fidelity.

Abstract

Generative semantic communication (Gen-SemCom) with large artificial intelligence (AI) model promises a transformative paradigm for 6G networks, which reduces communication costs by transmitting low-dimensional prompts rather than raw data. However, purely prompt-driven generation loses fine-grained visual details. Additionally, there is a lack of systematic metrics to evaluate the performance of Gen-SemCom systems. To address these issues, we develop a hybrid Gen-SemCom system with a critical information embedding (CIE) framework, where both text prompts and semantically critical features are extracted for transmissions. First, a novel approach of semantic filtering is proposed to select and transmit the semantically critical features of images relevant to semantic label. By integrating the text prompt and critical features, the receiver reconstructs high-fidelity images using a diffusion-based generative model. Next, we propose the generative visual information fidelity (GVIF) metric to evaluate the visual quality of the generated image. By characterizing the statistical models of image features, the GVIF metric quantifies the mutual information between the distorted features and their original counterparts. By maximizing the GVIF metric, we design a channel-adaptive Gen-SemCom system that adaptively control the volume of features and compression rate according to the channel state. Experimental results validate the GVIF metric's sensitivity to visual fidelity, correlating with both the PSNR and critical information volume. In addition, the optimized system achieves superior performance over benchmarking schemes in terms of higher PSNR and lower FID scores.