AgentComp: From Agentic Reasoning to Compositional Mastery in Text-to-Image Models

TL;DR

AgentComp introduces an innovative training framework that enhances text-to-image models' ability to understand and generate complex, compositional prompts by explicitly differentiating similar compositions, leading to improved accuracy and generalization.

Contribution

The paper presents a novel method using large language models and agentic preference optimization to improve compositional reasoning in text-to-image models, achieving state-of-the-art results.

Findings

State-of-the-art on T2I-CompBench

Improved differentiation of similar compositions

Maintains image quality while enhancing reasoning

Abstract

Text-to-image generative models have achieved remarkable visual quality but still struggle with compositionality$-$accurately capturing object relationships, attribute bindings, and fine-grained details in prompts. A key limitation is that models are not explicitly trained to differentiate between compositionally similar prompts and images, resulting in outputs that are close to the intended description yet deviate in fine-grained details. To address this, we propose AgentComp, a framework that explicitly trains models to better differentiate such compositional variations and enhance their reasoning ability. AgentComp leverages the reasoning and tool-use capabilities of large language models equipped with image generation, editing, and VQA tools to autonomously construct compositional datasets. Using these datasets, we apply an agentic preference optimization method to fine-tune text-to-image models, enabling them to better distinguish between compositionally similar samples and resulting in overall stronger compositional generation ability. AgentComp achieves state-of-the-art results on compositionality benchmarks such as T2I-CompBench, without compromising image quality$-$a common drawback in prior approaches$-$and even generalizes to other capabilities not explicitly trained for, such as text rendering.