GaussianCut: Interactive segmentation via graph cut for 3D Gaussian Splatting

TL;DR

GaussianCut is a novel interactive segmentation method for 3D scenes represented as Gaussians, using graph cuts and user input to efficiently partition objects without extra training.

Contribution

It introduces a graph-based segmentation approach that leverages 3D Gaussian Splatting and user interactions, eliminating the need for segmentation-aware training.

Findings

Achieves competitive performance with state-of-the-art methods

Works effectively across diverse scenes

Does not require additional segmentation-aware training

Abstract

We introduce GaussianCut, a new method for interactive multiview segmentation of scenes represented as 3D Gaussians. Our approach allows for selecting the objects to be segmented by interacting with a single view. It accepts intuitive user input, such as point clicks, coarse scribbles, or text. Using 3D Gaussian Splatting (3DGS) as the underlying scene representation simplifies the extraction of objects of interest which are considered to be a subset of the scene's Gaussians. Our key idea is to represent the scene as a graph and use the graph-cut algorithm to minimize an energy function to effectively partition the Gaussians into foreground and background. To achieve this, we construct a graph based on scene Gaussians and devise a segmentation-aligned energy function on the graph to combine user inputs with scene properties. To obtain an initial coarse segmentation, we leverage 2D image/video segmentation models and further refine these coarse estimates using our graph construction. Our empirical evaluations show the adaptability of GaussianCut across a diverse set of scenes. GaussianCut achieves competitive performance with state-of-the-art approaches for 3D segmentation without requiring any additional segmentation-aware training.