R3GW: Relightable 3D Gaussians for Outdoor Scenes in the Wild

TL;DR

R3GW introduces a relightable 3D Gaussian Splatting method for outdoor scenes in the wild, enabling photorealistic view synthesis under arbitrary lighting by modeling scene illumination explicitly.

Contribution

It presents a novel approach that separates scene components and models view-dependent lighting, allowing relighting of outdoor scenes captured in unconstrained conditions.

Findings

Achieves state-of-the-art relighting performance on NeRF-OSR dataset.

Improves rendering quality at sky-foreground boundaries.

Supports physically-based relighting in outdoor scenes.

Abstract

3D Gaussian Splatting (3DGS) has established itself as a leading technique for 3D reconstruction and novel view synthesis of static scenes, achieving outstanding rendering quality and fast training. However, the method does not explicitly model the scene illumination, making it unsuitable for relighting tasks. Furthermore, 3DGS struggles to reconstruct scenes captured in the wild by unconstrained photo collections featuring changing lighting conditions. In this paper, we present R3GW, a novel method that learns a relightable 3DGS representation of an outdoor scene captured in the wild. Our approach separates the scene into a relightable foreground and a non-reflective background (the sky), using two distinct sets of Gaussians. R3GW models view-dependent lighting effects in the foreground reflections by combining Physically Based Rendering with the 3DGS scene representation in a varying illumination setting. We evaluate our method quantitatively and qualitatively on the NeRF-OSR dataset, offering state-of-the-art performance and enhanced support for physically-based relighting of unconstrained scenes. Our method synthesizes photorealistic novel views under arbitrary illumination conditions. Additionally, our representation of the sky mitigates depth reconstruction artifacts, improving rendering quality at the sky-foreground boundary