Adaptive 3D Gaussian Splatting Video Streaming: Visual Saliency-Aware Tiling and Meta-Learning-Based Bitrate Adaptation

TL;DR

This paper introduces a comprehensive framework for 3D Gaussian splatting video streaming that includes saliency-guided tiling, a new quality assessment method, and a meta-learning-based bitrate adaptation, significantly improving streaming performance.

Contribution

It presents novel saliency-aware tiling, a joint quality assessment framework, and a meta-learning bitrate adaptation algorithm tailored for 3D Gaussian splatting videos.

Findings

Significant performance improvements over existing methods

Effective saliency-guided tiling enhances streaming quality

Meta-learning bitrate adaptation adapts well to network variability

Abstract

3D Gaussian splatting video (3DGS) streaming has recently emerged as a research hotspot in both academia and industry, owing to its impressive ability to deliver immersive 3D video experiences. However, research in this area is still in its early stages, and several fundamental challenges, such as tiling, quality assessment, and bitrate adaptation, require further investigation. In this paper, we tackle these challenges by proposing a comprehensive set of solutions. Specifically, we propose an adaptive 3DGS tiling technique guided by saliency analysis, which integrates both spatial and temporal features. Each tile is encoded into versions possessing dedicated deformation fields and multiple quality levels for adaptive selection. We also introduce a novel quality assessment framework for 3DGS video that jointly evaluates spatial-domain degradation in 3DGS representations during streaming and the quality of the resulting 2D rendered images. Additionally, we develop a meta-learning-based adaptive bitrate algorithm specifically tailored for 3DGS video streaming, achieving optimal performance across varying network conditions. Extensive experiments demonstrate that our proposed approaches significantly outperform state-of-the-art methods.