Retinex-Diffusion: On Controlling Illumination Conditions in Diffusion Models via Retinex Theory

TL;DR

This paper presents a novel method called Retinex-Diffusion that enables precise control of illumination conditions in diffusion models, producing realistic lighting effects without extra training or dataset requirements.

Contribution

It introduces a new approach to manipulate lighting in diffusion models by leveraging Retinex theory, avoiding the need for intrinsic decomposition or additional training.

Findings

Generates images with realistic shadows and reflections

Controls illumination properties during diffusion process

Does not require extra training or datasets

Abstract

This paper introduces a novel approach to illumination manipulation in diffusion models, addressing the gap in conditional image generation with a focus on lighting conditions. We conceptualize the diffusion model as a black-box image render and strategically decompose its energy function in alignment with the image formation model. Our method effectively separates and controls illumination-related properties during the generative process. It generates images with realistic illumination effects, including cast shadow, soft shadow, and inter-reflections. Remarkably, it achieves this without the necessity for learning intrinsic decomposition, finding directions in latent space, or undergoing additional training with new datasets.