PRTGS: Precomputed Radiance Transfer of Gaussian Splats for Real-Time High-Quality Relighting

TL;DR

PRTGS enables real-time, high-quality relighting of Gaussian splats in dynamic scenes by precomputing radiance transfer, capturing soft shadows and interreflections efficiently for immersive visual effects.

Contribution

It introduces a novel precomputing framework for radiance transfer in Gaussian splats, significantly improving real-time relighting quality and speed for complex, dynamic lighting environments.

Findings

Achieves state-of-the-art visual quality in relighting.

Maintains 30+ fps for dynamic scenes at 1080p.

Reduces training time with specialized precomputing methods.

Abstract

We proposed Precomputed RadianceTransfer of GaussianSplats (PRTGS), a real-time high-quality relighting method for Gaussian splats in low-frequency lighting environments that captures soft shadows and interreflections by precomputing 3D Gaussian splats' radiance transfer. Existing studies have demonstrated that 3D Gaussian splatting (3DGS) outperforms neural fields' efficiency for dynamic lighting scenarios. However, the current relighting method based on 3DGS still struggles to compute high-quality shadow and indirect illumination in real time for dynamic light, leading to unrealistic rendering results. We solve this problem by precomputing the expensive transport simulations required for complex transfer functions like shadowing, the resulting transfer functions are represented as dense sets of vectors or matrices for every Gaussian splat. We introduce distinct precomputing methods tailored for training and rendering stages, along with unique ray tracing and indirect lighting precomputation techniques for 3D Gaussian splats to accelerate training speed and compute accurate indirect lighting related to environment light. Experimental analyses demonstrate that our approach achieves state-of-the-art visual quality while maintaining competitive training times and allows high-quality real-time (30+ fps) relighting for dynamic light and relatively complex scenes at 1080p resolution.