Task-Oriented Image Semantic Communication Based on Rate-Distortion Theory

TL;DR

This paper introduces a task-oriented semantic communication scheme that optimizes both image quality and AI task performance by balancing pixel-level and semantic-level distortions using rate-distortion theory.

Contribution

It proposes a novel semantic communication framework with semantic reconstruction, formulating a new rate-distortion problem and deriving an analytical solution for optimal source mapping.

Findings

Outperforms traditional image codecs in reconstruction quality.

Enhances AI task performance and multi-task generalization.

Effectively balances image fidelity and semantic accuracy.

Abstract

Task-oriented image semantic communication is a new communication paradigm, which aims to transmit semantics for artificial intelligent (AI) tasks while ignoring the reconstruction quality of the images. However, in some applications, such as autonomous driving, both image reconstruction quality and the performance of the followed AI tasks must be simultaneously considered. To tackle this challenge, this paper proposes a task-oriented semantic communication scheme with semantic reconstruction (TOSC-SR). Its main goal is to simultaneously minimize pixel-level and task-relevant semantic-level distortion during communications under a certain rate, which formulates a new rate-distortion optimization problem. To successfully measure the loss at the semantic level, a new form of semantic distortion measured by the mutual information between the semantic-reconstructed images and the task labels is proposed. Then, we derive an analytical solution for the formulated problem, where the self-consistent equations of the problem are obtained to determine the optimal mapping of the source and the semantic-reconstructed images. To implement TOSC-SR, we further obtain an extended form of rate-distortion form based on the variational approximation of mutual information, which is applicable to multiple AI tasks. Experimental results show that the proposed approach outperforms the traditional JPEG, JPEG2000, BPG, VVC-based image communication systems and deep learning based benchmarks in terms of image reconstruction quality, AI task performance, and multi-task generalization ability.