GradiSeg: Gradient-Guided Gaussian Segmentation with Enhanced 3D Boundary Precision

TL;DR

GradiSeg introduces a gradient-guided 3D segmentation framework that enhances boundary accuracy by refining Gaussian distributions and adaptively propagating semantic information, improving 3D scene understanding and editing capabilities.

Contribution

The paper presents a novel 3D Gaussian segmentation method with Identity Gradient Guided Densification and Local Adaptive KNN modules, improving boundary recognition in 3D semantic segmentation.

Findings

Significant improvement in boundary segmentation accuracy.

Effective preservation of scene reconstruction quality.

Enhanced suitability for downstream scene editing tasks.

Abstract

While 3D Gaussian Splatting enables high-quality real-time rendering, existing Gaussian-based frameworks for 3D semantic segmentation still face significant challenges in boundary recognition accuracy. To address this, we propose a novel 3DGS-based framework named GradiSeg, incorporating Identity Encoding to construct a deeper semantic understanding of scenes. Our approach introduces two key modules: Identity Gradient Guided Densification (IGD) and Local Adaptive K-Nearest Neighbors (LA-KNN). The IGD module supervises gradients of Identity Encoding to refine Gaussian distributions along object boundaries, aligning them closely with boundary contours. Meanwhile, the LA-KNN module employs position gradients to adaptively establish locality-aware propagation of Identity Encodings, preventing irregular Gaussian spreads near boundaries. We validate the effectiveness of our method through comprehensive experiments. Results show that GradiSeg effectively addresses boundary-related issues, significantly improving segmentation accuracy without compromising scene reconstruction quality. Furthermore, our method's robust segmentation capability and decoupled Identity Encoding representation make it highly suitable for various downstream scene editing tasks, including 3D object removal, swapping and so on.