NG-GS: NeRF-Guided 3D Gaussian Splatting Segmentation

TL;DR

NG-GS introduces a novel framework for high-quality object segmentation in 3D Gaussian Splatting, explicitly addressing boundary discretization issues to improve segmentation accuracy and boundary quality.

Contribution

It proposes a boundary-aware segmentation method that uses mask variance analysis, RBF interpolation, and joint optimization with NeRF for smooth, accurate 3D segmentation.

Findings

Achieves state-of-the-art boundary mIoU on multiple benchmarks.

Effectively reduces artifacts and aliasing at object boundaries.

Demonstrates significant improvements over existing 3D segmentation methods.

Abstract

Recent advances in 3D Gaussian Splatting (3DGS) have enabled highly efficient and photorealistic novel view synthesis. However, segmenting objects accurately in 3DGS remains challenging due to the discrete nature of Gaussian representations, which often leads to aliasing and artifacts at object boundaries. In this paper, we introduce NG-GS, a novel framework for high-quality object segmentation in 3DGS that explicitly addresses boundary discretization. Our approach begins by automatically identifying ambiguous Gaussians at object boundaries using mask variance analysis. We then apply radial basis function (RBF) interpolation to construct a spatially continuous feature field, enhanced by multi-resolution hash encoding for efficient multi-scale representation. A joint optimization strategy aligns 3DGS with a lightweight NeRF module through alignment and spatial continuity losses, ensuring smooth and consistent segmentation boundaries. Extensive experiments on NVOS, LERF-OVS, and ScanNet benchmarks demonstrate that our method achieves state-of-the-art performance, with significant gains in boundary mIoU. Code is available at https://github.com/BJTU-KD3D/NG-GS.