Conquering High Packet-Loss Erasure: MoE Swin Transformer-Based Video Semantic Communication

TL;DR

This paper introduces a robust video semantic communication system using MoE Swin Transformer that effectively mitigates semantic information loss due to high packet loss, combining theoretical analysis and innovative recovery techniques.

Contribution

The paper proposes a novel MSTVSC system with packet-loss resistance, semantic recovery, and compression strategies, advancing robustness in packet-based semantic communication.

Findings

Achieves MS-SSIM > 0.6 at 90% packet loss

Maintains PSNR > 20 dB under high packet loss

Demonstrates superior performance over existing methods

Abstract

Semantic communication with joint semantic-channel coding robustly transmits diverse data modalities but faces challenges in mitigating semantic information loss due to packet drops in packet-based systems. Under current protocols, packets with errors are discarded, preventing the receiver from utilizing erroneous semantic data for robust decoding. To address this issue, a packet-loss-resistant MoE Swin Transformer-based Video Semantic Communication (MSTVSC) system is proposed in this paper. Semantic vectors are encoded by MSTVSC and transmitted through upper-layer protocol packetization. To investigate the impact of the packetization, a theoretical analysis of the packetization strategy is provided. To mitigate the semantic loss caused by packet loss, a 3D CNN at the receiver recovers missing information using un-lost semantic data and an packet-loss mask matrix. Semantic-level interleaving is employed to reduce concentrated semantic loss from packet drops. To improve compression, a common-individual decomposition approach is adopted, with downsampling applied to individual information to minimize redundancy. The model is lightweighted for practical deployment. Extensive simulations and comparisons demonstrate strong performance, achieving an MS-SSIM greater than 0.6 and a PSNR exceeding 20 dB at a 90% packet loss rate.