Training-Free Multi-View Extension of IC-Light for Textual Position-Aware Scene Relighting

TL;DR

GS-Light is a training-free, multi-view, text-guided relighting pipeline for 3D scenes that combines large vision-language models, geometry estimators, and diffusion models to produce high-fidelity relit scenes reflecting user prompts.

Contribution

It introduces a novel training-free extension of diffusion models for multi-view scene relighting guided by textual prompts, integrating lighting priors and view-geometry constraints.

Findings

Outperforms state-of-the-art baselines in multi-view consistency and image quality.

Produces high-fidelity, artistically relit 3D scenes from user prompts.

Demonstrates effectiveness on both indoor and outdoor scenes.

Abstract

We introduce GS-Light, an efficient, textual position-aware pipeline for text-guided relighting of 3D scenes represented via Gaussian Splatting (3DGS). GS-Light implements a training-free extension of a single-input diffusion model to handle multi-view inputs. Given a user prompt that may specify lighting direction, color, intensity, or reference objects, we employ a large vision-language model (LVLM) to parse the prompt into lighting priors. Using off-the-shelf estimators for geometry and semantics (depth, surface normals, and semantic segmentation), we fuse these lighting priors with view-geometry constraints to compute illumination maps and generate initial latent codes for each view. These meticulously derived init latents guide the diffusion model to generate relighting outputs that more accurately reflect user expectations, especially in terms of lighting direction. By feeding multi-view rendered images, along with the init latents, into our multi-view relighting model, we produce high-fidelity, artistically relit images. Finally, we fine-tune the 3DGS scene with the relit appearance to obtain a fully relit 3D scene. We evaluate GS-Light on both indoor and outdoor scenes, comparing it to state-of-the-art baselines including per-view relighting, video relighting, and scene editing methods. Using quantitative metrics (multi-view consistency, imaging quality, aesthetic score, semantic similarity, etc.) and qualitative assessment (user studies), GS-Light demonstrates consistent improvements over baselines. Code and assets will be made available upon publication.