Robust Prior-Guided Segmentation for Editable 3D Gaussian Splatting

TL;DR

This paper introduces a robust 3D segmentation framework using SAM-HQ and prior-guided label reassignment, enabling accurate, real-time object editing in 3D Gaussian Splatting scenes.

Contribution

It presents a novel method combining high-quality 2D masks and multiview consistent label reassignment for improved 3D segmentation accuracy.

Findings

Achieves state-of-the-art segmentation accuracy.

Enables interactive real-time object editing.

Maintains high visual fidelity in 3D scene editing.

Abstract

3D Gaussian Splatting (3D-GS) enables real-time 3D scene reconstruction but lacks robust segmentation for editing tasks such as object removal, extraction, and recoloring. Existing approaches that lift 2D segmentations to the 3D domain suffer from view inconsistencies and coarse masks. In this paper, we propose a novel framework that leverages the Segment Anything Model High Quality (SAM-HQ) to generate accurate 2D masks, addressing the limitations of the standard SAM in boundary fidelity and fine-structure preservation. To achieve robust 3D segmentation of any target object in a given scene, we introduce a prior-guided label reassignment method that assigns labels to 3D Gaussians by enforcing multiview consistency with learned priors. Our approach achieves state-of-the-art segmentation accuracy and enables interactive, real-time object editing while maintaining high visual fidelity. Qualitative results demonstrate superior boundary preservation and practical utility in Virtual Reality (VR) and robotics, advancing 3D scene editing.