LightSplat: Fast and Memory-Efficient Open-Vocabulary 3D Scene Understanding in Five Seconds

TL;DR

LightSplat is a fast, memory-efficient framework for open-vocabulary 3D scene understanding that avoids iterative optimization by using semantic indices and single-step clustering, enabling scalable language-driven 3D segmentation.

Contribution

We introduce LightSplat, a training-free method that significantly improves speed and memory efficiency in 3D scene understanding using semantic indices and clustering.

Findings

Achieves 50-400x speedup over existing methods

Uses 64x less memory compared to prior approaches

Maintains state-of-the-art accuracy in 3D segmentation

Abstract

Open-vocabulary 3D scene understanding enables users to segment novel objects in complex 3D environments through natural language. However, existing approaches remain slow, memory-intensive, and overly complex due to iterative optimization and dense per-Gaussian feature assignments. To address this, we propose LightSplat, a fast and memory-efficient training-free framework that injects compact 2-byte semantic indices into 3D representations from multi-view images. By assigning semantic indices only to salient regions and managing them with a lightweight index-feature mapping, LightSplat eliminates costly feature optimization and storage overhead. We further ensure semantic consistency and efficient inference via single-step clustering that links geometrically and semantically related masks in 3D. We evaluate our method on LERF-OVS, ScanNet, and DL3DV-OVS across complex indoor-outdoor scenes. As a result, LightSplat achieves state-of-the-art performance with up to 50-400x speedup and 64x lower memory, enabling scalable language-driven 3D understanding. For more details, visit our project page https://vision3d-lab.github.io/lightsplat/.