TexGS-VolVis: Expressive Scene Editing for Volume Visualization via Textured Gaussian Splatting

TL;DR

TexGS-VolVis introduces a textured Gaussian splatting framework for volume visualization that enables high-quality, style-flexible, and real-time scene editing using text and image guidance, surpassing existing methods.

Contribution

The paper presents TexGS-VolVis, a novel volumetric scene editing approach using textured Gaussian primitives and large models for flexible style transfer and fine-grained editing.

Findings

Outperforms existing methods in visual quality and efficiency

Enables real-time, style-flexible volume scene editing

Provides enhanced lighting control and partial editing capabilities

Abstract

Advancements in volume visualization (VolVis) focus on extracting insights from 3D volumetric data by generating visually compelling renderings that reveal complex internal structures. Existing VolVis approaches have explored non-photorealistic rendering techniques to enhance the clarity, expressiveness, and informativeness of visual communication. While effective, these methods often rely on complex predefined rules and are limited to transferring a single style, restricting their flexibility. To overcome these limitations, we advocate the representation of VolVis scenes using differentiable Gaussian primitives combined with pretrained large models to enable arbitrary style transfer and real-time rendering. However, conventional 3D Gaussian primitives tightly couple geometry and appearance, leading to suboptimal stylization results. To address this, we introduce TexGS-VolVis, a textured Gaussian splatting framework for VolVis. TexGS-VolVis employs 2D Gaussian primitives, extending each Gaussian with additional texture and shading attributes, resulting in higher-quality, geometry-consistent stylization and enhanced lighting control during inference. Despite these improvements, achieving flexible and controllable scene editing remains challenging. To further enhance stylization, we develop image- and text-driven non-photorealistic scene editing tailored for TexGS-VolVis and 2D-lift-3D segmentation to enable partial editing with fine-grained control. We evaluate TexGS-VolVis both qualitatively and quantitatively across various volume rendering scenes, demonstrating its superiority over existing methods in terms of efficiency, visual quality, and editing flexibility.