CASL: Concept-Aligned Sparse Latents for Interpreting Diffusion Models

TL;DR

CASL introduces a supervised framework that aligns sparse latent dimensions of diffusion models with semantic concepts, enabling more interpretable and controllable image generation.

Contribution

This work is the first to achieve supervised alignment of latent representations with semantic concepts in diffusion models, enhancing interpretability and control.

Findings

CASL achieves superior editing precision and interpretability.

CASL-Steer effectively reveals how concept latents influence generated content.

The method provides a causal probe for semantic understanding in diffusion models.

Abstract

Internal activations of diffusion models encode rich semantic information, but interpreting such representations remains challenging. While Sparse Autoencoders (SAEs) have shown promise in disentangling latent representations, existing SAE-based methods for diffusion model understanding rely on unsupervised approaches that fail to align sparse features with human-understandable concepts. This limits their ability to provide reliable semantic control over generated images. We introduce CASL (Concept-Aligned Sparse Latents), a supervised framework that aligns sparse latent dimensions of diffusion models with semantic concepts. CASL first trains an SAE on frozen U-Net activations to obtain disentangled latent representations, and then learns a lightweight linear mapping that associates each concept with a small set of relevant latent dimensions. To validate the semantic meaning of these aligned directions, we propose CASL-Steer, a controlled latent intervention that shifts activations along the learned concept axis. Unlike editing methods, CASL-Steer is used solely as a causal probe to reveal how concept-aligned latents influence generated content. We further introduce the Editing Precision Ratio (EPR), a metric that jointly measures concept specificity and the preservation of unrelated attributes. Experiments show that our method achieves superior editing precision and interpretability compared to existing approaches. To the best of our knowledge, this is the first work to achieve supervised alignment between latent representations and semantic concepts in diffusion models.