TAUE: Training-free Noise Transplant and Cultivation Diffusion Model

TL;DR

TAUE is a novel training-free diffusion model that enables layer-wise, coherent image generation with applications in editing and composition, matching the quality of fine-tuned models without additional data.

Contribution

It introduces a training-free framework that embeds structural and semantic information into diffusion models for layer-wise image control, eliminating the need for fine-tuning.

Findings

Achieves state-of-the-art performance among training-free methods.

Provides image quality comparable to fine-tuned models.

Enables applications like layout-aware editing and background replacement.

Abstract

Despite the remarkable success of text-to-image diffusion models, their output of a single, flattened image remains a critical bottleneck for professional applications requiring layer-wise control. Existing solutions either rely on fine-tuning with large, inaccessible datasets or are training-free yet limited to generating isolated foreground elements, failing to produce a complete and coherent scene. To address this, we introduce the Training-free Noise Transplantation and Cultivation Diffusion Model (TAUE), a novel framework for layer-wise image generation that requires neither fine-tuning nor additional data. TAUE embeds global structural information from intermediate denoising latents into the initial noise to preserve spatial coherence, and integrates semantic cues through cross-layer attention sharing to maintain contextual and visual consistency across layers. Extensive experiments demonstrate that TAUE achieves state-of-the-art performance among training-free methods, delivering image quality comparable to fine-tuned models while improving inter-layer consistency. Moreover, it enables new applications, such as layout-aware editing, multi-object composition, and background replacement, indicating potential for interactive, layer-separated generation systems in real-world creative workflows.