3D Gaussian Splatting with Deferred Reflection

TL;DR

This paper introduces a deferred shading technique for Gaussian splatting that effectively renders specular reflections, improving visual quality and normal estimation in novel view synthesis without sacrificing rendering speed.

Contribution

It proposes a novel deferred shading method that enhances specular reflection rendering in Gaussian radiance fields by leveraging reflection gradients for better normal estimation.

Findings

Outperforms state-of-the-art methods in specular reflection synthesis

Achieves higher PSNR on synthetic and real scenes

Maintains real-time rendering speeds

Abstract

The advent of neural and Gaussian-based radiance field methods have achieved great success in the field of novel view synthesis. However, specular reflection remains non-trivial, as the high frequency radiance field is notoriously difficult to fit stably and accurately. We present a deferred shading method to effectively render specular reflection with Gaussian splatting. The key challenge comes from the environment map reflection model, which requires accurate surface normal while simultaneously bottlenecks normal estimation with discontinuous gradients. We leverage the per-pixel reflection gradients generated by deferred shading to bridge the optimization process of neighboring Gaussians, allowing nearly correct normal estimations to gradually propagate and eventually spread over all reflective objects. Our method significantly outperforms state-of-the-art techniques and concurrent work in synthesizing high-quality specular reflection effects, demonstrating a consistent improvement of peak signal-to-noise ratio (PSNR) for both synthetic and real-world scenes, while running at a frame rate almost identical to vanilla Gaussian splatting.