AirGS: Real-Time 4D Gaussian Streaming for Free-Viewpoint Video Experiences

TL;DR

AirGS is a streaming-optimized 4D Gaussian Splatting framework that enhances free-viewpoint video quality and efficiency by improving training, transmission, and reconstruction processes for real-time immersive experiences.

Contribution

It introduces a novel streaming pipeline, keyframe enhancement, and adaptive pruning algorithms to improve 4DGS quality and reduce bandwidth in free-viewpoint videos.

Findings

Reduces PSNR quality deviation by over 20% during scene changes.

Maintains frame PSNR above 30 consistently.

Speeds up training by 6 times and halves transmission size.

Abstract

Free-viewpoint video (FVV) enables immersive viewing experiences by allowing users to view scenes from arbitrary perspectives. As a prominent reconstruction technique for FVV generation, 4D Gaussian Splatting (4DGS) models dynamic scenes with time-varying 3D Gaussian ellipsoids and achieves high-quality rendering via fast rasterization. However, existing 4DGS approaches suffer from quality degradation over long sequences and impose substantial bandwidth and storage overhead, limiting their applicability in real-time and wide-scale deployments. Therefore, we present AirGS, a streaming-optimized 4DGS framework that rearchitects the training and delivery pipeline to enable high-quality, low-latency FVV experiences. AirGS converts Gaussian video streams into multi-channel 2D formats and intelligently identifies keyframes to enhance frame reconstruction quality. It further combines temporal coherence with inflation loss to reduce training time and representation size. To support communication-efficient transmission, AirGS models 4DGS delivery as an integer linear programming problem and design a lightweight pruning level selection algorithm to adaptively prune the Gaussian updates to be transmitted, balancing reconstruction quality and bandwidth consumption. Extensive experiments demonstrate that AirGS reduces quality deviation in PSNR by more than 20% when scene changes, maintains frame-level PSNR consistently above 30, accelerates training by 6 times, reduces per-frame transmission size by nearly 50% compared to the SOTA 4DGS approaches.