LabelGS: Label-Aware 3D Gaussian Splatting for 3D Scene Segmentation

TL;DR

LabelGS enhances 3D Gaussian Splatting with label-aware segmentation, enabling accurate scene understanding and achieving significant speedup over previous methods.

Contribution

It introduces a novel label-aware approach for 3D Gaussian Splatting, including new models and strategies for efficient and accurate 3D scene segmentation.

Findings

Outperforms previous state-of-the-art in 3D scene segmentation.

Achieves 22X faster training speed at high resolution.

Effectively decouples Gaussian representations for improved segmentation.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a novel explicit representation for 3D scenes, offering both high-fidelity reconstruction and efficient rendering. However, 3DGS lacks 3D segmentation ability, which limits its applicability in tasks that require scene understanding. The identification and isolating of specific object components is crucial. To address this limitation, we propose Label-aware 3D Gaussian Splatting (LabelGS), a method that augments the Gaussian representation with object label.LabelGS introduces cross-view consistent semantic masks for 3D Gaussians and employs a novel Occlusion Analysis Model to avoid overfitting occlusion during optimization, Main Gaussian Labeling model to lift 2D semantic prior to 3D Gaussian and Gaussian Projection Filter to avoid Gaussian label conflict. Our approach achieves effective decoupling of Gaussian representations and refines the 3DGS optimization process through a random region sampling strategy, significantly improving efficiency. Extensive experiments demonstrate that LabelGS outperforms previous state-of-the-art methods, including Feature-3DGS, in the 3D scene segmentation task. Notably, LabelGS achieves a remarkable 22X speedup in training compared to Feature-3DGS, at a resolution of 1440X1080. Our code will be at https://github.com/garrisonz/LabelGS.