Wireless Video Semantic Communication with Decoupled Diffusion Multi-frame Compensation

TL;DR

This paper introduces WVSC-D, a wireless video transmission framework that encodes videos at the semantic level and employs a diffusion-based compensation method, significantly improving bandwidth efficiency and video quality.

Contribution

It proposes a novel semantic-level video coding framework with a decoupled diffusion multi-frame compensation method, reducing communication overhead and enhancing transmission performance.

Findings

WVSC-D outperforms existing DL-based methods like DVSC by about 1.8 dB PSNR.

Semantic encoding reduces bandwidth compared to pixel-level coding.

Diffusion-based compensation improves video quality at lower bitrates.

Abstract

Existing wireless video transmission schemes directly conduct video coding in pixel level, while neglecting the inner semantics contained in videos. In this paper, we propose a wireless video semantic communication framework with decoupled diffusion multi-frame compensation (DDMFC), abbreviated as WVSC-D, which integrates the idea of semantic communication into wireless video transmission scenarios. WVSC-D first encodes original video frames as semantic frames and then conducts video coding based on such compact representations, enabling the video coding in semantic level rather than pixel level. Moreover, to further reduce the communication overhead, a reference semantic frame is introduced to substitute motion vectors of each frame in common video coding methods. At the receiver, DDMFC is proposed to generate compensated current semantic frame by a two-stage conditional diffusion process. With both the reference frame transmission and DDMFC frame compensation, the bandwidth efficiency improves with satisfying video transmission performance. Experimental results verify the performance gain of WVSC-D over other DL-based methods e.g. DVSC about 1.8 dB in terms of PSNR.