IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing

TL;DR

IRGS introduces a differentiable 2D Gaussian ray tracing method for inverse rendering, enabling accurate modeling of inter-reflections and indirect radiance, surpassing previous approximations and simplifying complex light interactions.

Contribution

The paper presents a novel inter-reflective Gaussian splatting method with a differentiable 2D Gaussian ray tracer for improved inverse rendering accuracy.

Findings

Accurately models complex inter-reflections in inverse rendering.

Outperforms previous methods on standard benchmarks.

Efficient optimization scheme for Monte Carlo sampling.

Abstract

In inverse rendering, accurately modeling visibility and indirect radiance for incident light is essential for capturing secondary effects. Due to the absence of a powerful Gaussian ray tracer, previous 3DGS-based methods have either adopted a simplified rendering equation or used learnable parameters to approximate incident light, resulting in inaccurate material and lighting estimations. To this end, we introduce inter-reflective Gaussian splatting (IRGS) for inverse rendering. To capture inter-reflection, we apply the full rendering equation without simplification and compute incident radiance on the fly using the proposed differentiable 2D Gaussian ray tracing. Additionally, we present an efficient optimization scheme to handle the computational demands of Monte Carlo sampling for rendering equation evaluation. Furthermore, we introduce a novel strategy for querying the indirect radiance of incident light when relighting the optimized scenes. Extensive experiments on multiple standard benchmarks validate the effectiveness of IRGS, demonstrating its capability to accurately model complex inter-reflection effects.