Style Composition within Distinct LoRA modules for Traditional Art

TL;DR

This paper introduces a zero-shot diffusion method that enables controlled, region-specific style blending in images by combining multiple style-trained models and using depth-map conditioning for structural coherence.

Contribution

It presents a novel style composition technique within diffusion models that allows smooth, region-specific style mixing using separate style models and spatial masks, with depth conditioning for coherence.

Findings

Effective region-specific style blending demonstrated

Preserves individual style fidelity during mixing

Depth conditioning improves structural coherence

Abstract

Diffusion-based text-to-image models have achieved remarkable results in synthesizing diverse images from text prompts and can capture specific artistic styles via style personalization. However, their entangled latent space and lack of smooth interpolation make it difficult to apply distinct painting techniques in a controlled, regional manner, often causing one style to dominate. To overcome this, we propose a zero-shot diffusion pipeline that naturally blends multiple styles by performing style composition on the denoised latents predicted during the flow-matching denoising process of separately trained, style-specialized models. We leverage the fact that lower-noise latents carry stronger stylistic information and fuse them across heterogeneous diffusion pipelines using spatial masks, enabling precise, region-specific style control. This mechanism preserves the fidelity of each individual style while allowing user-guided mixing. Furthermore, to ensure structural coherence across different models, we incorporate depth-map conditioning via ControlNet into the diffusion framework. Qualitative and quantitative experiments demonstrate that our method successfully achieves region-specific style mixing according to the given masks.