Goal-Oriented Framework for Optical Flow-based Multi-User Multi-Task Video Transmission

TL;DR

This paper introduces a goal-oriented semantic communication framework for multi-user multi-task video transmission using optical flow, improving video quality, classification accuracy, and bandwidth efficiency in wireless systems.

Contribution

The paper presents a novel OF-GSC framework with a semantic encoder, transformer decoder, and DDPG-based bandwidth allocation, enhancing multi-task video transmission performance.

Findings

13.47% increase in SSIM for video reconstruction

Top-1 accuracy surpassing VideoMAE with 25% data

Bandwidth reduction of 25.97% with DDPG algorithm

Abstract

Efficient multi-user multi-task video transmission is an important research topic within the realm of current wireless communication systems. To reduce the transmission burden and save communication resources, we propose a goal-oriented semantic communication framework for optical flow-based multi-user multi-task video transmission (OF-GSC). At the transmitter, we design a semantic encoder that consists of a motion extractor and a patch-level optical flow-based semantic representation extractor to effectively identify and select important semantic representations. At the receiver, we design a transformer-based semantic decoder for high-quality video reconstruction and video classification tasks. To minimize the communication time, we develop a deep deterministic policy gradient (DDPG)-based bandwidth allocation algorithm for multi-user transmission. For video reconstruction tasks, our OF-GSC framework achieves a significant improvement in the received video quality, as evidenced by a 13.47% increase in the structural similarity index measure (SSIM) score in comparison to DeepJSCC. For video classification tasks, OF-GSC achieves a Top-1 accuracy slightly surpassing the performance of VideoMAE with only 25% required data under the same mask ratio of 0.3. For bandwidth allocation optimization, our DDPG-based algorithm reduces the maximum transmission time by 25.97% compared with the baseline equal-bandwidth allocation scheme.