RNG: Relightable Neural Gaussians

TL;DR

RNG introduces a novel 3D Gaussian Splatting framework that enables relighting of complex objects without relying on traditional shading assumptions, achieving fast training and high-quality shadows.

Contribution

The paper presents RNG, a relightable 3D Gaussian Splatting method that handles objects with diverse surface types and improves shadow accuracy without shading model assumptions.

Findings

Faster training time of 1.3 hours

Real-time rendering at 60 fps

Higher-quality shadows than prior methods

Abstract

3D Gaussian Splatting (3DGS) has shown impressive results for the novel view synthesis task, where lighting is assumed to be fixed. However, creating relightable 3D assets, especially for objects with ill-defined shapes (fur, fabric, etc.), remains a challenging task. The decomposition between light, geometry, and material is ambiguous, especially if either smooth surface assumptions or surfacebased analytical shading models do not apply. We propose Relightable Neural Gaussians (RNG), a novel 3DGS-based framework that enables the relighting of objects with both hard surfaces or soft boundaries, while avoiding assumptions on the shading model. We condition the radiance at each point on both view and light directions. We also introduce a shadow cue, as well as a depth refinement network to improve shadow accuracy. Finally, we propose a hybrid forward-deferred fitting strategy to balance geometry and appearance quality. Our method achieves significantly faster training (1.3 hours) and rendering (60 frames per second) compared to a prior method based on neural radiance fields and produces higher-quality shadows than a concurrent 3DGS-based method. Project page: https://www.whois-jiahui.fun/project_pages/RNG.