FreeControl: Efficient, Training-Free Structural Control via One-Step Attention Extraction

TL;DR

FreeControl offers a training-free, efficient method for structural control in diffusion models by extracting one-step attention, enabling flexible, high-quality image generation without retraining or inversion.

Contribution

It introduces a novel one-step attention extraction method and Latent-Condition Decoupling, improving control, quality, and stability in diffusion-based image synthesis without additional training.

Findings

Achieves structural control with approximately 5% extra inference cost.

Supports compositional control using multiple reference images.

Eliminates structural artifacts and improves stability through LCD.

Abstract

Controlling the spatial and semantic structure of diffusion-generated images remains a challenge. Existing methods like ControlNet rely on handcrafted condition maps and retraining, limiting flexibility and generalization. Inversion-based approaches offer stronger alignment but incur high inference cost due to dual-path denoising. We present FreeControl, a training-free framework for semantic structural control in diffusion models. Unlike prior methods that extract attention across multiple timesteps, FreeControl performs one-step attention extraction from a single, optimally chosen key timestep and reuses it throughout denoising. This enables efficient structural guidance without inversion or retraining. To further improve quality and stability, we introduce Latent-Condition Decoupling (LCD): a principled separation of the key timestep and the noised latent used in attention extraction. LCD provides finer control over attention quality and eliminates structural artifacts. FreeControl also supports compositional control via reference images assembled from multiple sources - enabling intuitive scene layout design and stronger prompt alignment. FreeControl introduces a new paradigm for test-time control, enabling structurally and semantically aligned, visually coherent generation directly from raw images, with the flexibility for intuitive compositional design and compatibility with modern diffusion models at approximately 5 percent additional cost.