A Unified Multi-Task Semantic Communication System for Multimodal Data

TL;DR

This paper introduces U-DeepSC, a unified multi-task semantic communication system that efficiently handles multimodal data, adapts to different tasks and channel conditions, and reduces transmission overhead and model size.

Contribution

The paper proposes a novel unified deep learning framework for multi-task semantic communication, featuring a vector-wise dynamic scheme, a lightweight feature selection module, and a shared codebook.

Findings

Achieves comparable performance to task-specific systems

Reduces transmission overhead significantly

Lowers model size while maintaining accuracy

Abstract

Task-oriented semantic communications have achieved significant performance gains. However, the employed deep neural networks in semantic communications have to be updated when the task is changed or multiple models need to be stored for performing different tasks. To address this issue, we develop a unified deep learning-enabled semantic communication system (U-DeepSC), where a unified end-to-end framework can serve many different tasks with multiple modalities of data. As the number of required features varies from task to task, we propose a vector-wise dynamic scheme that can adjust the number of transmitted symbols for different tasks. Moreover, our dynamic scheme can also adaptively adjust the number of transmitted features under different channel conditions to optimize the transmission efficiency. Particularly, we devise a lightweight feature selection module (FSM) to evaluate the importance of feature vectors, which can hierarchically drop redundant feature vectors and significantly accelerate the inference. To reduce the transmission overhead, we then design a unified codebook for feature representation to serve multiple tasks, where only the indices of these task-specific features in the codebook are transmitted. According to the simulation results, the proposed U-DeepSC achieves comparable performance to the task-oriented semantic communication system designed for a specific task but with significant reduction in both transmission overhead and model size.