TR-Gaussians: High-fidelity Real-time Rendering of Planar Transmission and Reflection with 3D Gaussian Splatting

TL;DR

TR-Gaussians introduces a novel 3D Gaussian-based method for real-time, high-fidelity rendering of planar transmission and reflection effects in indoor scenes, effectively modeling view-dependent appearance using learnable reflection planes.

Contribution

The paper presents TR-Gaussians, a new 3D Gaussian representation that explicitly models planar transmission and reflection with view-dependent effects, enabling real-time high-fidelity rendering.

Findings

Achieves real-time rendering of scenes with planar transmission and reflection.

Outperforms existing methods quantitatively and qualitatively.

Effectively models view-dependent appearance using Fresnel-based blending.

Abstract

We propose Transmission-Reflection Gaussians (TR-Gaussians), a novel 3D-Gaussian-based representation for high-fidelity rendering of planar transmission and reflection, which are ubiquitous in indoor scenes. Our method combines 3D Gaussians with learnable reflection planes that explicitly model the glass planes with view-dependent reflectance strengths. Real scenes and transmission components are modeled by 3D Gaussians and the reflection components are modeled by the mirrored Gaussians with respect to the reflection plane. The transmission and reflection components are blended according to a Fresnel-based, view-dependent weighting scheme, allowing for faithful synthesis of complex appearance effects under varying viewpoints. To effectively optimize TR-Gaussians, we develop a multi-stage optimization framework incorporating color and geometry constraints and an opacity perturbation mechanism. Experiments on different datasets demonstrate that TR-Gaussians achieve real-time, high-fidelity novel view synthesis in scenes with planar transmission and reflection, and outperform state-of-the-art approaches both quantitatively and qualitatively.