Interpretable Diffusion Models with B-cos Networks

TL;DR

This paper introduces B-cos diffusion models that enhance interpretability in text-to-image generation by linking prompt tokens to specific image regions, improving understanding of prompt-image alignment.

Contribution

The paper presents a novel diffusion model architecture with B-cos modules that inherently provides interpretability in image generation.

Findings

High-quality image generation with interpretability

Token-to-region explanations for prompt influence

Improved understanding of prompt-image alignment

Abstract

Text-to-image diffusion models generate images by iteratively denoising random noise, conditioned on a prompt. While these models have enabled impressive progress in image generation, they often fail to accurately reflect all semantic information described in the prompt -- failures that are difficult to detect automatically. In this work, we introduce a diffusion model architecture built with B-cos modules that offers inherent interpretability. Our approach provides insight into how individual prompt tokens affect the generated image by producing explanations that highlight the pixel regions influenced by each token. We demonstrate that B-cos diffusion models can produce high-quality images while providing meaningful insights into prompt-image alignment.