CSGaussian: Progressive Rate-Distortion Compression and Segmentation for 3D Gaussian Splatting

TL;DR

This paper introduces CSGaussian, a unified framework that combines rate-distortion optimized compression with semantic segmentation for 3D Gaussian Splatting, enabling efficient scene editing and understanding.

Contribution

It integrates semantic learning into 3D Gaussian Splatting compression using a lightweight neural hyperprior, enabling joint compression and segmentation for the first time.

Findings

Significantly reduces transmission costs

Maintains high rendering quality

Achieves strong segmentation performance

Abstract

We present the first unified framework for rate-distortion-optimized compression and segmentation of 3D Gaussian Splatting (3DGS). While 3DGS has proven effective for both real-time rendering and semantic scene understanding, prior works have largely treated these tasks independently, leaving their joint consideration unexplored. Inspired by recent advances in rate-distortion-optimized 3DGS compression, this work integrates semantic learning into the compression pipeline to support decoder-side applications--such as scene editing and manipulation--that extend beyond traditional scene reconstruction and view synthesis. Our scheme features a lightweight implicit neural representation-based hyperprior, enabling efficient entropy coding of both color and semantic attributes while avoiding costly grid-based hyperprior as seen in many prior works. To facilitate compression and segmentation, we further develop compression-guided segmentation learning, consisting of quantization-aware training to enhance feature separability and a quality-aware weighting mechanism to suppress unreliable Gaussian primitives. Extensive experiments on the LERF and 3D-OVS datasets demonstrate that our approach significantly reduces transmission cost while preserving high rendering quality and strong segmentation performance.