2D-Guided 3D Gaussian Segmentation

TL;DR

This paper introduces a novel 3D Gaussian segmentation method guided by 2D segmentation maps, enabling efficient multi-object segmentation with competitive accuracy and improved simplicity over existing approaches.

Contribution

The paper presents a new 3D Gaussian segmentation approach that leverages 2D segmentation supervision, simplifying multi-object segmentation in 3D scenes.

Findings

Achieves comparable mIOU and mAcc to previous methods

Enables multi-object segmentation in 3D efficiently

Uses 2D segmentation maps to guide 3D Gaussian learning

Abstract

Recently, 3D Gaussian, as an explicit 3D representation method, has demonstrated strong competitiveness over NeRF (Neural Radiance Fields) in terms of expressing complex scenes and training duration. These advantages signal a wide range of applications for 3D Gaussians in 3D understanding and editing. Meanwhile, the segmentation of 3D Gaussians is still in its infancy. The existing segmentation methods are not only cumbersome but also incapable of segmenting multiple objects simultaneously in a short amount of time. In response, this paper introduces a 3D Gaussian segmentation method implemented with 2D segmentation as supervision. This approach uses input 2D segmentation maps to guide the learning of the added 3D Gaussian semantic information, while nearest neighbor clustering and statistical filtering refine the segmentation results. Experiments show that our concise method can achieve comparable performances on mIOU and mAcc for multi-object segmentation as previous single-object segmentation methods.