RT-GS: Gaussian Splatting with Reflection and Transmittance Primitives

TL;DR

RT-GS introduces a unified Gaussian Splatting framework that models specular reflections and transmittance using separate primitives and differentiable ray tracing, enabling realistic rendering of complex scenes.

Contribution

It combines a microfacet model with ray tracing in Gaussian Splatting to jointly simulate reflections and transparency, addressing limitations of previous methods.

Findings

Successfully models reflections and objects behind transparent surfaces.

Achieves significant qualitative improvements over prior methods.

Demonstrates effectiveness in complex environments with specular interactions.

Abstract

Gaussian Splatting is a powerful tool for reconstructing diffuse scenes, but it struggles to simultaneously model specular reflections and the appearance of objects behind semi-transparent surfaces. These specular reflections and transmittance are essential for realistic novel view synthesis, and existing methods do not properly incorporate the underlying physical processes to simulate them. To address this issue, we propose RT-GS, a unified framework that integrates a microfacet material model and ray tracing to jointly model specular reflection and transmittance in Gaussian Splatting. We accomplish this by using separate Gaussian primitives for reflections and transmittance, which allow modeling distant reflections and reconstructing objects behind transparent surfaces concurrently. We utilize a differentiable ray tracing framework to obtain the specular reflection and transmittance appearance. Our experiments demonstrate that our method successfully produces reflections and recovers objects behind transparent surfaces in complex environments, achieving significant qualitative improvements over prior methods where these specular light interactions are prominent.