GUS-IR: Gaussian Splatting with Unified Shading for Inverse Rendering

TL;DR

GUS-IR introduces a unified shading framework for inverse rendering that effectively handles complex scenes with rough and glossy surfaces, improving shape reconstruction and intrinsic decomposition for applications like relighting and retouching.

Contribution

The paper proposes a novel unified shading approach combining forward and deferred shading, along with enhanced normal modeling and ambient occlusion techniques, to improve inverse rendering of complex scenes.

Findings

Superior intrinsic decomposition accuracy

Enhanced geometric shape reconstruction

Effective handling of indirect illumination

Abstract

Recovering the intrinsic physical attributes of a scene from images, generally termed as the inverse rendering problem, has been a central and challenging task in computer vision and computer graphics. In this paper, we present GUS-IR, a novel framework designed to address the inverse rendering problem for complicated scenes featuring rough and glossy surfaces. This paper starts by analyzing and comparing two prominent shading techniques popularly used for inverse rendering, forward shading and deferred shading, effectiveness in handling complex materials. More importantly, we propose a unified shading solution that combines the advantages of both techniques for better decomposition. In addition, we analyze the normal modeling in 3D Gaussian Splatting (3DGS) and utilize the shortest axis as normal for each particle in GUS-IR, along with a depth-related regularization, resulting in improved geometric representation and better shape reconstruction. Furthermore, we enhance the probe-based baking scheme proposed by GS-IR to achieve more accurate ambient occlusion modeling to better handle indirect illumination. Extensive experiments have demonstrated the superior performance of GUS-IR in achieving precise intrinsic decomposition and geometric representation, supporting many downstream tasks (such as relighting, retouching) in computer vision, graphics, and extended reality.