RT-Splatting: Joint Reflection-Transmission Modeling with Gaussian Splatting

TL;DR

RT-Splatting introduces a novel scene representation that disentangles reflection and transmission in 3D Gaussian Splatting, enabling high-quality, real-time rendering of semi-transparent scenes with improved visual fidelity.

Contribution

It proposes a unified surface-volume scene representation with Gaussian primitives and a hybrid renderer, along with Specular-Aware Gradient Gating to improve semi-transparent scene rendering.

Findings

Achieves state-of-the-art results on semi-transparent scenes.

Delivers high-fidelity reflections and clear transmission in real-time.

Enables flexible scene editing through factorization.

Abstract

3D Gaussian Splatting (3DGS) enables real-time novel view synthesis with high visual quality. However, existing methods struggle with semi-transparent specular surfaces that exhibit both complex reflections and clear transmission, often producing blurry reflections or overly occluded transmission. To address this, we present RT-Splatting, a framework that disentangles each Gaussian's geometric occupancy from its optical opacity. This factorization yields a unified surface-volume scene representation with a single set of Gaussian primitives. Our hybrid renderer interprets this representation both as a surface to capture high-frequency reflections and as a volume to preserve clear transmission. To mitigate the ambiguity in jointly optimizing reflection and transmission, we introduce Specular-Aware Gradient Gating, which suppresses misleading gradients from highly specular regions into the transmission branch, effectively reducing distracting floaters. Experiments on challenging semi-transparent scenes show that RT-Splatting achieves state-of-the-art performance, delivering high-fidelity reflections and clear transmission with real-time rendering. Moreover, our factorization naturally enables flexible scene editing. The project page is available at https://sjj118.github.io/RT-Splatting.