Temporal Concept Dynamics in Diffusion Models via Prompt-Conditioned Interventions

TL;DR

This paper introduces PCI, a training-free framework to analyze how concepts form and stabilize during the diffusion process in text-to-image models, enabling better understanding and control of image generation trajectories.

Contribution

The paper presents PCI, a novel, model-agnostic method for studying concept dynamics in diffusion models through temporal interventions without retraining.

Findings

Different diffusion models exhibit diverse concept formation timings.

Certain phases in the diffusion process are more conducive to specific concepts.

PCI enables more effective and semantically accurate image editing.

Abstract

Diffusion models are usually evaluated by their final outputs, gradually denoising random noise into meaningful images. Yet, generation unfolds along a trajectory, and analyzing this dynamic process is crucial for understanding how controllable, reliable, and predictable these models are in terms of their success/failure modes. In this work, we ask the question: when does noise turn into a specific concept (e.g., age) and lock in the denoising trajectory? We propose PCI (Prompt-Conditioned Intervention) to study this question. PCI is a training-free and model-agnostic framework for analyzing concept dynamics through diffusion time. The central idea is the analysis of Concept Insertion Success (CIS), defined as the probability that a concept inserted at a given timestep is preserved and reflected in the final image, offering a way to characterize the temporal dynamics of concept formation. Applied to several state-of-the-art text-to-image diffusion models and a broad taxonomy of concepts, PCI reveals diverse temporal behaviors across diffusion models, in which certain phases of the trajectory are more favorable to specific concepts even within the same concept type. These findings also provide actionable insights for text-driven image editing, highlighting when interventions are most effective without requiring access to model internals or training, and yielding quantitatively stronger edits that achieve a balance of semantic accuracy and content preservation than strong baselines. Code is available at: https://adagorgun.github.io/PCI-Project/