LuxRemix: Lighting Decomposition and Remixing for Indoor Scenes

TL;DR

LuxRemix introduces a real-time, interactive method for decomposing and relighting indoor scenes from a single multi-view capture, enabling independent control of light sources with high photorealism.

Contribution

It presents a novel generative image-based lighting decomposition model integrated with multi-view harmonization and relightable 3D Gaussian splatting for interactive indoor scene editing.

Findings

Achieves photorealistic lighting decomposition and relighting.

Demonstrates real-time interactive control over individual light sources.

Outperforms state-of-the-art techniques on synthetic and real datasets.

Abstract

We present a novel approach for interactive light editing in indoor scenes from a single multi-view scene capture. Our method leverages a generative image-based light decomposition model that factorizes complex indoor scene illumination into its constituent light sources. This factorization enables independent manipulation of individual light sources, specifically allowing control over their state (on/off), chromaticity, and intensity. We further introduce multi-view lighting harmonization to ensure consistent propagation of the lighting decomposition across all scene views. This is integrated into a relightable 3D Gaussian splatting representation, providing real-time interactive control over the individual light sources. Our results demonstrate highly photorealistic lighting decomposition and relighting outcomes across diverse indoor scenes. We evaluate our method on both synthetic and real-world datasets and provide a quantitative and qualitative comparison to state-of-the-art techniques. For video results and interactive demos, see https://luxremix.github.io.