HPC: Hierarchical Point-based Latent Representation for Streaming Dynamic Gaussian Splatting Compression

TL;DR

HPC introduces a hierarchical point-based latent compression framework for streaming dynamic Gaussian Splatting, significantly reducing storage needs while preserving high-quality rendering in free-viewpoint videos.

Contribution

It proposes a novel hierarchical point-based latent representation and neural network compression method, addressing redundancy and compactness issues in existing streaming Gaussian Splatting techniques.

Findings

Achieves 67% storage reduction compared to baseline.

Maintains high reconstruction fidelity.

Outperforms state-of-the-art methods in experiments.

Abstract

While dynamic Gaussian Splatting has driven significant advances in free-viewpoint video, maintaining its rendering quality with a small memory footprint for efficient streaming transmission still presents an ongoing challenge. Existing streaming dynamic Gaussian Splatting compression methods typically leverage a latent representation to drive the neural network for predicting Gaussian residuals between frames. Their core latent representations can be categorized into structured grid-based and unstructured point-based paradigms. However, the former incurs significant parameter redundancy by inevitably modeling unoccupied space, while the latter suffers from limited compactness as it fails to exploit local correlations. To relieve these limitations, we propose HPC, a novel streaming dynamic Gaussian Splatting compression framework. It employs a hierarchical point-based latent representation that operates on a per-Gaussian basis to avoid parameter redundancy in unoccupied space. Guided by a tailored aggregation scheme, these latent points achieve high compactness with low spatial redundancy. To improve compression efficiency, we further undertake the first investigation to compress neural networks for streaming dynamic Gaussian Splatting through mining and exploiting the inter-frame correlation of parameters. Combined with latent compression, this forms a fully end-to-end compression framework. Comprehensive experimental evaluations demonstrate that HPC substantially outperforms state-of-the-art methods. It achieves a storage reduction of 67% against its baseline while maintaining high reconstruction fidelity.