Task-Oriented Multi-User Semantic Communications

TL;DR

This paper develops Transformer-based multi-user semantic communication systems for single-modal and multimodal data, demonstrating improved robustness, efficiency, and task performance over traditional methods.

Contribution

It introduces a unified Transformer framework for multi-user semantic communications across different tasks and modalities, including novel models for image retrieval, translation, and VQA.

Findings

Models outperform traditional methods in robustness and efficiency.

DeepSC-IR and DeepSC-MT effectively optimize semantic embedding and meaning recovery.

DeepSC-VQA successfully fuses multimodal data for visual question answering.

Abstract

While semantic communications have shown the potential in the case of single-modal single-users, its applications to the multi-user scenario remain limited. In this paper, we investigate deep learning (DL) based multi-user semantic communication systems for transmitting single-modal data and multimodal data, respectively. We will adopt three intelligent tasks, including, image retrieval, machine translation, and visual question answering (VQA) as the transmission goal of semantic communication systems. We will then propose a Transformer based unique framework to unify the structure of transmitters for different tasks. For the single-modal multi-user system, we will propose two Transformer based models, named, DeepSC-IR and DeepSC-MT, to perform image retrieval and machine translation, respectively. In this case, DeepSC-IR is trained to optimize the distance in embedding space between images and DeepSC-MT is trained to minimize the semantic errors by recovering the semantic meaning of sentences. For the multimodal multi-user system, we develop a Transformer enabled model, named, DeepSC-VQA, for the VQA task by extracting text-image information at the transmitters and fusing it at the receiver. In particular, a novel layer-wise Transformer is designed to help fuse multimodal data by adding connection between each of the encoder and decoder layers. Numerical results will show that the proposed models are superior to traditional communications in terms of the robustness to channels, computational complexity, transmission delay, and the task-execution performance at various task-specific metrics.