CoMPaSS: Enhancing Spatial Understanding in Text-to-Image Diffusion Models

TL;DR

CoMPaSS significantly improves spatial accuracy in text-to-image diffusion models by curating better training data and enhancing token order understanding, leading to state-of-the-art results on spatial benchmarks.

Contribution

The paper introduces CoMPaSS, a novel framework combining data curation and token encoding techniques to enhance spatial understanding in T2I models.

Findings

Achieves +98% on VISOR benchmark

Improves T2I-CompBench Spatial by +67%

Increases GenEval Position accuracy by +131%

Abstract

Text-to-image (T2I) diffusion models excel at generating photorealistic images but often fail to render accurate spatial relationships. We identify two core issues underlying this common failure: 1) the ambiguous nature of data concerning spatial relationships in existing datasets, and 2) the inability of current text encoders to accurately interpret the spatial semantics of input descriptions. We propose CoMPaSS, a versatile framework that enhances spatial understanding in T2I models. It first addresses data ambiguity with the Spatial Constraints-Oriented Pairing (SCOP) data engine, which curates spatially-accurate training data via principled constraints. To leverage these priors, CoMPaSS also introduces the Token ENcoding ORdering (TENOR) module, which preserves crucial token ordering information lost by text encoders, thereby reinforcing the prompt's linguistic structure. Extensive experiments on four popular T2I models (UNet and MMDiT-based) show CoMPaSS sets a new state of the art on key spatial benchmarks, with substantial relative gains on VISOR (+98%), T2I-CompBench Spatial (+67%), and GenEval Position (+131%). Code is available at https://github.com/blurgyy/CoMPaSS.