QoE Optimization for Semantic Self-Correcting Video Transmission in Multi-UAV Networks

TL;DR

This paper introduces a semantic self-correcting video transmission framework for UAV networks that enhances bandwidth efficiency and robustness against packet loss by combining semantic encoding, multi-frame decoding, and reinforcement learning-based QoE optimization.

Contribution

It proposes a novel semantic encoding and adaptive transmission framework (SSCV-G) with reinforcement learning (MUPPO) for improved UAV video streaming performance.

Findings

Outperforms state-of-the-art codecs in coding efficiency

Enhances bandwidth adaptability and packet loss robustness

Achieves superior QoE optimization through reinforcement learning

Abstract

Real-time unmanned aerial vehicle (UAV) video streaming is essential for time-sensitive applications, including remote surveillance, emergency response, and environmental monitoring. However, it faces challenges such as limited bandwidth, latency fluctuations, and high packet loss. To address these issues, we propose a novel semantic self-correcting video transmission framework with ultra-fine bitrate granularity (SSCV-G). In SSCV-G, video frames are encoded into a compact semantic codebook space, and the transmitter adaptively sends a subset of semantic indices based on bandwidth availability, enabling fine-grained bitrate control for improved bandwidth efficiency. At the receiver, a spatio-temporal vision transformer (ST-ViT) performs multi-frame joint decoding to reconstruct dropped semantic indices by modeling intra- and inter-frame dependencies. To further improve performance under dynamic network conditions, we integrate a multi-user proximal policy optimization (MUPPO) reinforcement learning scheme that jointly optimizes communication resource allocation and semantic bitrate selection to maximize user Quality of Experience (QoE). Extensive experiments demonstrate that the proposed SSCV-G significantly outperforms state-of-the-art video codecs in coding efficiency, bandwidth adaptability, and packet loss robustness. Moreover, the proposed MUPPO-based QoE optimization consistently surpasses existing benchmarks.