InnerGS: Internal Scenes Reconstruction and Segmentation via Factorized 3D Gaussian Splatting

TL;DR

InnerGS introduces a novel method for reconstructing and segmenting internal 3D scenes using factorized Gaussian splatting, enabling detailed interior modeling and text-guided segmentation without camera poses.

Contribution

The paper presents a new approach for internal scene reconstruction using continuous volumetric density modeling with 3D Gaussians, extending Gaussian splatting to internal structures and incorporating language features for segmentation.

Findings

Effective reconstruction of internal structures from sparse data.

Enables text-guided segmentation of internal scenes.

No requirement for camera pose information.

Abstract

3D Gaussian Splatting (3DGS) has recently gained popularity for efficient scene rendering by representing scenes as explicit sets of anisotropic 3D Gaussians. However, most existing work focuses primarily on modeling external surfaces. In this work, we target the reconstruction of internal scenes, which is crucial for applications that require a deep understanding of an object's interior. By directly modeling a continuous volumetric density through the inner 3D Gaussian distribution, our model effectively reconstructs smooth and detailed internal structures from sparse sliced data. Beyond high-fidelity reconstruction, we further demonstrate the framework's potential for downstream tasks such as segmentation. By integrating language features, we extend our approach to enable text-guided segmentation of medical scenes via natural language queries. Our approach eliminates the need for camera poses, is plug-and-play, and is inherently compatible with any data modalities. We provide cuda implementation at: https://github.com/Shuxin-Liang/InnerGS.