FlashSplat: 2D to 3D Gaussian Splatting Segmentation Solved Optimally

TL;DR

This paper introduces a globally optimal, fast, and robust linear programming-based method for 3D Gaussian Splatting segmentation from 2D masks, significantly outperforming existing iterative approaches in speed and accuracy.

Contribution

We propose a novel linear programming solution for 3D Gaussian Splatting segmentation that is both globally optimal and computationally efficient, addressing limitations of iterative methods.

Findings

Optimization completes within 30 seconds, 50 times faster than previous methods.

Method demonstrates superior robustness against noise in segmentation tasks.

Achieves better performance in downstream applications like object removal and inpainting.

Abstract

This study addresses the challenge of accurately segmenting 3D Gaussian Splatting from 2D masks. Conventional methods often rely on iterative gradient descent to assign each Gaussian a unique label, leading to lengthy optimization and sub-optimal solutions. Instead, we propose a straightforward yet globally optimal solver for 3D-GS segmentation. The core insight of our method is that, with a reconstructed 3D-GS scene, the rendering of the 2D masks is essentially a linear function with respect to the labels of each Gaussian. As such, the optimal label assignment can be solved via linear programming in closed form. This solution capitalizes on the alpha blending characteristic of the splatting process for single step optimization. By incorporating the background bias in our objective function, our method shows superior robustness in 3D segmentation against noises. Remarkably, our optimization completes within 30 seconds, about 50$\times$ faster than the best existing methods. Extensive experiments demonstrate the efficiency and robustness of our method in segmenting various scenes, and its superior performance in downstream tasks such as object removal and inpainting. Demos and code will be available at https://github.com/florinshen/FlashSplat.