GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization

TL;DR

This paper introduces GIR, a novel 3D Gaussian inverse rendering method that factorizes scenes into material, light, and geometry, enabling real-time relighting and view synthesis with state-of-the-art accuracy.

Contribution

The paper proposes a new 3D Gaussian representation for inverse rendering, including normal estimation, indirect illumination tracing, and environmental map learning, achieving real-time performance.

Findings

State-of-the-art relighting accuracy

Real-time rendering capability

Effective scene factorization

Abstract

This paper presents a 3D Gaussian Inverse Rendering (GIR) method, employing 3D Gaussian representations to effectively factorize the scene into material properties, light, and geometry. The key contributions lie in three-fold. We compute the normal of each 3D Gaussian using the shortest eigenvector, with a directional masking scheme forcing accurate normal estimation without external supervision. We adopt an efficient voxel-based indirect illumination tracing scheme that stores direction-aware outgoing radiance in each 3D Gaussian to disentangle secondary illumination for approximating multi-bounce light transport. To further enhance the illumination disentanglement, we represent a high-resolution environmental map with a learnable low-resolution map and a lightweight, fully convolutional network. Our method achieves state-of-the-art performance in both relighting and novel view synthesis tasks among the recently proposed inverse rendering methods while achieving real-time rendering. This substantiates our proposed method's efficacy and broad applicability, highlighting its potential as an influential tool in various real-time interactive graphics applications such as material editing and relighting. The code will be released at https://github.com/guduxiaolang/GIR.