The Entropic Signature of Class Speciation in Diffusion Models

TL;DR

This paper introduces an entropy-based method to detect and analyze the transition from semantic ambiguity to class commitment in diffusion models, linking information theory with diffusion dynamics.

Contribution

It proposes tracking class-conditional entropy as a signature of transition regimes and demonstrates its effectiveness on real diffusion models, connecting theory with practical detection.

Findings

Entropy rate concentrates on a logarithmic time scale.

Entropy-based signatures reliably identify semantic transition regimes.

Guidance redistributes semantic information over time.

Abstract

Diffusion models do not recover semantic structure uniformly over time. Instead, samples transition from semantic ambiguity to class commitment within a narrow regime. Recent theoretical work attributes this transition to dynamical instabilities along class-separating directions, but practical methods to detect and exploit these windows in trained models are still limited. We show that tracking the class-conditional entropy of a latent semantic variable given the noisy state provides a reliable signature of these transition regimes. By restricting the entropy to semantic partitions, the entropy can furthermore resolve semantic decisions at different levels of abstraction. We analyze this behavior in high-dimensional Gaussian mixture models and show that the entropy rate concentrates on the same logarithmic time scale as the speciation symmetry-breaking instability previously identified in variance-preserving diffusion. We validate our method on EDM2-XS and Stable Diffusion 1.5, where class-conditional entropy consistently isolates the noise regimes critical for semantic structure formation. Finally, we use our framework to quantify how guidance redistributes semantic information over time. Together, these results connect information-theoretic and statistical physics perspectives on diffusion and provide a principled basis for time-localized control.