Stochastic Ray Tracing for the Reconstruction of 3D Gaussian Splatting

TL;DR

This paper introduces a stochastic, differentiable ray tracing method for 3D Gaussian splatting that improves reconstruction quality and speed, enabling more accurate relightable scene rendering without sorting all intersecting Gaussians.

Contribution

It presents the first unbiased Monte Carlo estimator for differentiable, sorting-free stochastic ray tracing in 3D Gaussian splatting, enhancing efficiency and fidelity.

Findings

Matches rasterization speed and quality for standard 3DGS

Outperforms sorting-based ray tracing in reconstruction quality

Achieves higher fidelity in relightable scene rendering

Abstract

Ray-tracing-based 3D Gaussian splatting (3DGS) methods overcome the limitations of rasterization -- rigid pinhole camera assumptions, inaccurate shadows, and lack of native reflection or refraction -- but remain slower due to the cost of sorting all intersecting Gaussians along every ray. Moreover, existing ray-tracing methods still rely on rasterization-style approximations such as shadow mapping for relightable scenes, undermining the generality that ray tracing promises. We present a differentiable, sorting-free stochastic formulation for ray-traced 3DGS -- the first framework that uses stochastic ray tracing to both reconstruct and render standard and relightable 3DGS scenes. At its core is an unbiased Monte Carlo estimator for pixel-color gradients that evaluates only a small sampled subset of Gaussians per ray, bypassing the need for sorting. For standard 3DGS, our method matches the reconstruction quality and speed of rasterization-based 3DGS while substantially outperforming sorting-based ray tracing. For relightable 3DGS, the same stochastic estimator drives per-Gaussian shading with fully ray-traced shadow rays, delivering notably higher reconstruction fidelity than prior work.