Right Looks, Wrong Reasons: Compositional Fidelity in Text-to-Image Generation

TL;DR

This paper highlights the failure of current text-to-image models to handle logical composition, especially with negation, counting, and spatial relations, due to data, architecture, and evaluation limitations.

Contribution

It provides a comprehensive analysis of compositional failures in text-to-image models and identifies key factors behind this issue, emphasizing the need for fundamental advances.

Findings

Models fail on combined primitives despite accuracy on individual ones

Training data lacks explicit negations, affecting compositional understanding

Current architectures and metrics are inadequate for logical reasoning

Abstract

The architectural blueprint of today's leading text-to-image models contains a fundamental flaw: an inability to handle logical composition. This survey investigates this breakdown across three core primitives-negation, counting, and spatial relations. Our analysis reveals a dramatic performance collapse: models that are accurate on single primitives fail precipitously when these are combined, exposing severe interference. We trace this failure to three key factors. First, training data show a near-total absence of explicit negations. Second, continuous attention architectures are fundamentally unsuitable for discrete logic. Third, evaluation metrics reward visual plausibility over constraint satisfaction. By analyzing recent benchmarks and methods, we show that current solutions and simple scaling cannot bridge this gap. Achieving genuine compositionality, we conclude, will require fundamental advances in representation and reasoning rather than incremental adjustments to existing architectures.