Few Channels Draw The Whole Picture: Revealing Massive Activations in Diffusion Transformers

TL;DR

This paper reveals that a small subset of massive activations in diffusion transformers critically influence image generation, organize spatial semantics, and can be transferred across prompts for controllable image synthesis.

Contribution

It uncovers the functional, spatial, and transfer properties of massive activations, highlighting their role as a sparse semantic carrier in diffusion transformer models.

Findings

Massive activations are functionally critical for image quality.

They are spatially organized and align with main image subjects.

Transferring massive activations enables prompt interpolation and subject-driven generation.

Abstract

Diffusion Transformers (DiTs) and related flow-based architectures are now among the strongest text-to-image generators, yet the internal mechanisms through which prompts shape image semantics remain poorly understood. In this work, we study massive activations: a small subset of hidden-state channels whose responses are consistently much larger than the rest. We show that, despite their sparsity, these few channels effectively draw the whole picture, in three complementary senses. First, they are functionally critical: a controlled disruption probe that zeroes the massive channels causes a sharp collapse in generation quality, while disrupting an equally-sized set of low-statistic channels has marginal effect. Second, they are spatially organized: restricting image-stream tokens to massive channels and clustering them yields coherent partitions that closely align with the main subject and salient regions, exposing a structured spatial code hidden inside an apparently outlier-like subspace. Third, they are transferable: transporting massive activations from one prompt-conditioned trajectory into another, shifts the final image toward the source prompt while preserving substantial content from the target, producing localized semantic interpolation rather than unstructured pixel blending. We exploit this property in two use cases: text-conditioned and image-conditioned semantic transport, where massive activations transport enables prompt interpolation and subject-driven generation without any additional training. Together, these results recast massive activations not as activation anomalies, but as a sparse prompt-conditioned carrier subspace that organizes and controls semantic information in modern DiT models.