MVLight: Relightable Text-to-3D Generation via Light-conditioned Multi-View Diffusion

TL;DR

MVLight is a novel multi-view diffusion model that incorporates lighting conditions to generate high-quality, relightable 3D models from text descriptions, improving geometric accuracy and lighting fidelity.

Contribution

The paper introduces MVLight, a light-conditioned diffusion model that explicitly models lighting for better 3D generation and relighting from text prompts.

Findings

Enhanced relighting capabilities demonstrated

Improved geometric precision in generated models

Validated effectiveness through experiments and user study

Abstract

Recent advancements in text-to-3D generation, building on the success of high-performance text-to-image generative models, have made it possible to create imaginative and richly textured 3D objects from textual descriptions. However, a key challenge remains in effectively decoupling light-independent and lighting-dependent components to enhance the quality of generated 3D models and their relighting performance. In this paper, we present MVLight, a novel light-conditioned multi-view diffusion model that explicitly integrates lighting conditions directly into the generation process. This enables the model to synthesize high-quality images that faithfully reflect the specified lighting environment across multiple camera views. By leveraging this capability to Score Distillation Sampling (SDS), we can effectively synthesize 3D models with improved geometric precision and relighting capabilities. We validate the effectiveness of MVLight through extensive experiments and a user study.