Semantic Foam: Unifying Spatial and Semantic Scene Decomposition

TL;DR

Semantic Foam enhances 3D scene decomposition by integrating semantic features into Radiant Foam's volumetric mesh, improving segmentation quality and consistency for interactive graphics applications.

Contribution

It introduces a novel semantic decomposition method that combines Radiant Foam's spatial mesh with explicit semantic features, addressing segmentation challenges in 3D scene representations.

Findings

Achieves superior object-level segmentation compared to Gaussian Grouping and SAGA.

Prevents artifacts caused by occlusion and inconsistent supervision.

Enables direct spatial regularization for better segmentation quality.

Abstract

Modern scene reconstruction methods, such as 3D Gaussian Splatting, deliver photo-realistic novel view synthesis at real-time speeds, yet their adoption in interactive graphics applications has been limited. A major bottleneck is the difficulty of interacting with these representations compared to traditional, human-authored 3D assets. While previous research has attempted to impose semantic decomposition on these models, significant challenges remain regarding segmentation quality and consistency. To address this, we introduce Semantic Foam, extending the recently proposed Radiant Foam representations to semantic decomposition tasks. Our approach integrates the natural spatial volumetric decomposition of Radiant Foam's Voronoi mesh with an explicit semantic feature field parameterized at the cell level. This explicit structure enables direct spatial regularization, which prevents artifacts caused by occlusion or inconsistent supervision across views - common pitfalls for other point-based representations. Experimental results show that our method achieves superior object-level segmentation performance compared to state-of-the-art methods like Gaussian Grouping and SAGA.