Semantic Communications for Digital Signals via Carrier Images

TL;DR

This paper introduces a novel semantic communication method that encodes digital signals into images using mask locations, leveraging pre-trained Masked Autoencoders to efficiently transmit digital data alongside images with minimal overhead.

Contribution

It proposes a new SemCom framework that encodes digital signals as masked regions on images and utilizes MAE for joint image and digital signal transmission, reducing communication costs.

Findings

Maintains reliable digital signal transmission at high mask ratios.

Reduces transmission overhead by encoding mask tokens as sparse matrices.

Uses pre-trained MAE for efficient encoding and decoding.

Abstract

Most of current semantic communication (SemCom) frameworks focus on the image transmission, which, however, do not address the problem on how to deliver digital signals without any semantic features. This paper proposes a novel SemCom approach to transmit digital signals by using the image as the carrier signal. Specifically, the proposed approach encodes the digital signal as a binary stream and maps it to mask locations on an image. This allows binary data to be visually represented, enabling the use of existing model, pre-trained Masked Autoencoders (MAE), which are optimized for masked image reconstruction, as the SemCom encoder and decoder. Since MAE can both process and recover masked images, this approach allows for the joint transmission of digital signals and images without incurring significant communication overheads. In addition, considering the mask tokens transmission encoded by the MAE still faces extra costs, we design a sparse encoding module at the transmitter to encode the mask tokens into a sparse matrix, and it can be recovered at the receiver. Thus, this approach simply needs to transmit the latent representations of the unmasked patches and a sparse matrix, which further reduce the transmission overhead compared with the original MAE encoder. Simulation results show that the approach maintains reliable transmission even in a high mask ratio of images.