RelightMaster: Precise Video Relighting with Multi-plane Light Images
Weikang Bian, Xiaoyu Shi, Zhaoyang Huang, Jianhong Bai, Qinghe Wang, Xintao Wang, Pengfei Wan, Kun Gai, Hongsheng Li
TL;DR
RelightMaster introduces a novel framework for precise, controllable video relighting by creating a new dataset, modeling lighting with multi-plane light images, and integrating this into pre-trained diffusion models for realistic results.
Contribution
The paper presents RelightMaster, a new approach combining a specialized dataset, multi-plane light image modeling, and a light image adapter to enable accurate and controllable video relighting.
Findings
Generates physically plausible lighting and shadows.
Preserves original scene content during relighting.
Outperforms existing methods in relighting quality.
Abstract
Recent advances in diffusion models enable high-quality video generation and editing, but precise relighting with consistent video contents, which is critical for shaping scene atmosphere and viewer attention, remains unexplored. Mainstream text-to-video (T2V) models lack fine-grained lighting control due to text's inherent limitation in describing lighting details and insufficient pre-training on lighting-related prompts. Additionally, constructing high-quality relighting training data is challenging, as real-world controllable lighting data is scarce. To address these issues, we propose RelightMaster, a novel framework for accurate and controllable video relighting. First, we build RelightVideo, the first dataset with identical dynamic content under varying precise lighting conditions based on the Unreal Engine. Then, we introduce Multi-plane Light Image (MPLI), a novel visual prompt…
Peer Reviews
Decision·Submitted to ICLR 2026
- The most interesting aspect of this paper is the proposed Multi-plane Light Image (MPLI) for lighting representation. It is extended from the idea of the Multi-Plane Image for 3D representation and adapted to lighting. Such a representation could be useful for a broader range of applications that utilize lighting. - The RelightVideo dataset could be a good contribution, if it will be released to the public.
- While the idea of the Multi-plane Light Image is interesting, the current usage of it to represent lighting is limited. Take a look at Figures 3 and 4, the majority of the MPLI are empty (i.e., black). This seems to be a not-so-efficient way of representing *point lights*. MPLI should be able to capture much complex spatially and temporally varying lighting. I feel like this is a missed opportunity. - There is no quantitative evaluation in the experiment section. I understand that evaluatin
1. The paper proposes MPLI representation effectively encodes 3D light source properties and naturally aligns with video modality, enabling precise and dynamic multi-light control. 2. The Light Image Adapter offers a lightweight and efficient integration strategy that preserves pre-trained model knowledge and avoids catastrophic forgetting. 3. The RelightVideo dataset provides the first large-scale video relighting dataset with consistent dynamic content under varied lighting conditions.
1. The experimental section lacks quantitative results and metrics, relying solely on qualitative visual examples, which makes a rigorous performance comparison impossible. 2. The comparison with state-of-the-art methods is limited to only two other approaches (Light-A-Video and TC-Light) and relies solely on qualitative visual examples, with no quantitative metrics or user studies provided to substantiate the claimed superiority. 3. The method's performance and generalization on real-world vide
1. Addresses a clear gap in fine-grained video relighting 2. Physically interpretable representation of lighting through MPLI 3. The use of synthetic paired data is well justified. 4. The framework demonstrates convincing qualitative results for position, color, intensity and depth control.
1. Directional modeling: The paper assumes isotropic point lights defined by position, color, and intensity only. Modeling directional or area lights would improve realism and generality. Future work should explore non-isotropic light sources. 2. Quantitative evaluation: No numerical comparison is provided. It would be important to include metrics such as (a) relighted image quality (FID or CLIP-based), (b) temporal consistency, and (c) user preference. Qualitative evaluation alone is insuffici
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Computer Graphics and Visualization Techniques · Image Enhancement Techniques