Deep Joint Source-Channel Coding Based on Semantics of Pixels

TL;DR

This paper introduces a deep joint source-channel coding approach that preserves semantic information of images for intelligent tasks during wireless transmission, improving classification performance.

Contribution

It proposes a novel semantic-aware deep coding method that minimizes semantic distortion, outperforming existing methods in intelligent image classification tasks.

Findings

Improves classification accuracy by 1.38% over SOTA methods.

Achieves 66% better performance than traditional coding methods.

Effectively preserves semantic information during transmission.

Abstract

The semantic information of the image for intelligent tasks is hidden behind the pixels, and slight changes in the pixels will affect the performance of intelligent tasks. In order to preserve semantic information behind pixels for intelligent tasks during wireless image transmission, we propose a joint source-channel coding method based on semantics of pixels, which can improve the performance of intelligent tasks for images at the receiver by retaining semantic information. Specifically, we first utilize gradients of intelligent task's perception results with respect to pixels to represent the semantic importance of pixels. Then, we extract the semantic distortion, and train the deep joint source-channel coding network with the goal of minimizing semantic distortion rather than pixel's distortion. Experiment results demonstrate that the proposed method improves the performance of the intelligent classification task by 1.38% and 66% compared with the SOTA deep joint source-channel coding method and the traditional separately source-channel coding method at the same transmission ra te and signal-to-noise ratio.