Tile-Weighted Rate-Distortion Optimized Packet Scheduling for 360$^\circ$ VR Video Streaming

TL;DR

This paper introduces a tile-weighted rate-distortion optimization system for 360° VR video streaming that dynamically weights tiles based on predicted viewpoints to enhance quality and reduce data usage.

Contribution

It proposes a novel TWRD packet scheduling method using a multimodal transformer for viewpoint prediction and dynamic programming for optimization.

Findings

Outperforms existing methods in reducing data volume.

Improves video quality under limited bandwidth.

Effective viewpoint prediction enhances scheduling accuracy.

Abstract

A key challenge of 360$^\circ$ VR video streaming is ensuring high quality with limited network bandwidth. Currently, most studies focus on tile-based adaptive bitrate streaming to reduce bandwidth consumption, where resources in network nodes are not fully utilized. This article proposes a tile-weighted rate-distortion (TWRD) packet scheduling optimization system to reduce data volume and improve video quality. A multimodal spatial-temporal attention transformer is proposed to predict viewpoint with probability that is used to dynamically weight tiles and corresponding packets. The packet scheduling problem of determining which packets should be dropped is formulated as an optimization problem solved by a dynamic programming solution. Experiment results demonstrate the proposed method outperforms the existing methods under various conditions.