StreamSTGS: Streaming Spatial and Temporal Gaussian Grids for Real-Time Free-Viewpoint Video
Zhihui Ke, Yuyang Liu, Xiaobo Zhou, Tie Qiu

TL;DR
StreamSTGS is a novel real-time streaming free-viewpoint video method that significantly reduces storage requirements and enables adaptive bitrate control, achieving competitive quality with much smaller frame sizes.
Contribution
The paper introduces StreamSTGS, a new FVV representation that allows real-time streaming with high compression and adaptive bitrate, supporting local and global motion learning.
Findings
Increases PSNR by an average of 1dB over state-of-the-art methods.
Reduces average frame size to 170KB, enabling efficient streaming.
Demonstrates competitive performance on diverse FVV benchmarks.
Abstract
Streaming free-viewpoint video~(FVV) in real-time still faces significant challenges, particularly in training, rendering, and transmission efficiency. Harnessing superior performance of 3D Gaussian Splatting~(3DGS), recent 3DGS-based FVV methods have achieved notable breakthroughs in both training and rendering. However, the storage requirements of these methods can reach up to MB per frame, making stream FVV in real-time impossible. To address this problem, we propose a novel FVV representation, dubbed StreamSTGS, designed for real-time streaming. StreamSTGS represents a dynamic scene using canonical 3D Gaussians, temporal features, and a deformation field. For high compression efficiency, we encode canonical Gaussian attributes as 2D images and temporal features as a video. This design not only enables real-time streaming, but also inherently supports adaptive bitrate control…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsVideo Coding and Compression Technologies · Advanced Vision and Imaging · Image and Video Quality Assessment
