Delta-K: Boosting Multi-Instance Generation via Cross-Attention Augmentation

TL;DR

Delta-K is a novel inference framework that enhances multi-instance scene synthesis in diffusion models by injecting semantic signals into the cross-attention space, improving concept coherence without retraining.

Contribution

The paper introduces Delta-K, a plug-and-play method that operates in the shared cross-attention key space to address concept omission in diffusion models, without additional training or architectural changes.

Findings

Improves compositional alignment across various diffusion architectures.

Operates without spatial masks or retraining.

Enhances semantic coherence in complex scenes.

Abstract

While Diffusion Models excel in text-to-image synthesis, they often suffer from concept omission when synthesizing complex multi-instance scenes. Existing training-free methods attempt to resolve this by rescaling attention maps, which merely exacerbates unstructured noise without establishing coherent semantic representations. To address this, we propose Delta-K, a backbone-agnostic and plug-and-play inference framework that tackles omission by operating directly in the shared cross-attention Key space. Specifically, with Vision-language model, we extract a differential key $\Delta K$ that encodes the semantic signature of missing concepts. This signal is then injected during the early semantic planning stage of the diffusion process. Governed by a dynamically optimized scheduling mechanism, Delta-K grounds diffuse noise into stable structural anchors while preserving existing concepts. Extensive experiments demonstrate the generality of our approach: Delta-K consistently improves compositional alignment across both modern DiT models and classical U-Net architectures, without requiring spatial masks, additional training, or architectural modifications.