Region-Adaptive Learned Hierarchical Encoding for 3D Gaussian Splatting Data

TL;DR

This paper presents RALHE, a hierarchical encoding method for 3D Gaussian Splatting data that significantly reduces model size while maintaining high-quality rendering, enabling bandwidth-efficient 3D scene streaming.

Contribution

We propose a novel learned hierarchical latent encoding tailored for 3D Gaussian Splatting data, leveraging octree structures and autoregressive models for efficient compression.

Findings

Achieves up to 2dB PSNR gain at low bitrates (<1MB)

Reduces model size for 3DGS data significantly

Improves bandwidth efficiency for 3D scene streaming

Abstract

We introduce Region-Adaptive Learned Hierarchical Encoding (RALHE) for 3D Gaussian Splatting (3DGS) data. While 3DGS has recently become popular for novel view synthesis, the size of trained models limits its deployment in bandwidth-constrained applications such as volumetric media streaming. To address this, we propose a learned hierarchical latent representation that builds upon the principles of "overfitted" learned image compression (e.g., Cool-Chic and C3) to efficiently encode 3DGS attributes. Unlike images, 3DGS data have irregular spatial distributions of Gaussians (geometry) and consist of multiple attributes (signals) defined on the irregular geometry. Our codec is designed to account for these differences between images and 3DGS. Specifically, we leverage the octree structure of the voxelized 3DGS geometry to obtain a hierarchical multi-resolution representation. Our approach overfits latents to each Gaussian attribute under a global rate constraint. These latents are decoded independently through a lightweight decoder network. To estimate the bitrate during training, we employ an autoregressive probability model that leverages octree-derived contexts from the 3D point structure. The multi-resolution latents, decoder, and autoregressive entropy coding networks are jointly optimized for each Gaussian attribute. Experiments demonstrate that the proposed RALHE compression framework achieves a rendering PSNR gain of up to 2dB at low bitrates (less than 1 MB) compared to the baseline 3DGS compression methods.