Generalizable and Relightable Gaussian Splatting for Human Novel View Synthesis

TL;DR

GRGS is a novel 3D Gaussian framework enabling high-fidelity, relightable human view synthesis that generalizes across characters and lighting by integrating geometry, material, and illumination cues with physics-based rendering.

Contribution

It introduces a fully supervised, generalizable approach with a lighting-robust geometry refinement and physically grounded neural rendering for editable relighting.

Findings

Achieves superior visual quality and geometric consistency.

Generalizes well across diverse characters and lighting conditions.

Supports editable relighting with shadows and indirect illumination.

Abstract

We propose GRGS, a generalizable and relightable 3D Gaussian framework for high-fidelity human novel view synthesis under diverse lighting conditions. Unlike existing methods that rely on per-character optimization or ignore physical constraints, GRGS adopts a feed-forward, fully supervised strategy projecting geometry, material, and illumination cues from multi-view 2D observations into 3D Gaussian representations. To recover accurate geometry under diverse lighting conditions, we introduce a Lighting-robust Geometry Refinement (LGR) module trained on synthetically relit data to predict precise depth and surface normals. Based on the high-quality geometry, a Physically Grounded Neural Rendering (PGNR) module is further proposed to integrate neural prediction with physics-based shading, supporting editable relighting with shadows and indirect illumination. Moreover, we design a 2D-to-3D projection training scheme leveraging differentiable supervision from ambient occlusion, direct, and indirect lighting maps, alleviating the computational cost of ray tracing. Extensive experiments demonstrate that GRGS achieves superior visual quality, geometric consistency, and generalization across characters and lighting conditions.