Draw ALL Your Imagine: A Holistic Benchmark and Agent Framework for Complex Instruction-based Image Generation

TL;DR

This paper introduces LongBench-T2I, a comprehensive benchmark for evaluating text-to-image models on complex instructions, and proposes an agent framework, Plan2Gen, that improves image generation fidelity without retraining models.

Contribution

The paper presents a new benchmark for complex instruction-based image generation and an agent framework that enhances generation quality by decomposing prompts, without additional training.

Findings

LongBench-T2I includes 500 prompts across nine visual dimensions.

Plan2Gen effectively guides existing T2I models on complex prompts.

New evaluation toolkit captures nuances of complex instruction adherence.

Abstract

Recent advancements in text-to-image (T2I) generation have enabled models to produce high-quality images from textual descriptions. However, these models often struggle with complex instructions involving multiple objects, attributes, and spatial relationships. Existing benchmarks for evaluating T2I models primarily focus on general text-image alignment and fail to capture the nuanced requirements of complex, multi-faceted prompts. Given this gap, we introduce LongBench-T2I, a comprehensive benchmark specifically designed to evaluate T2I models under complex instructions. LongBench-T2I consists of 500 intricately designed prompts spanning nine diverse visual evaluation dimensions, enabling a thorough assessment of a model's ability to follow complex instructions. Beyond benchmarking, we propose an agent framework (Plan2Gen) that facilitates complex instruction-driven image generation without requiring additional model training. This framework integrates seamlessly with existing T2I models, using large language models to interpret and decompose complex prompts, thereby guiding the generation process more effectively. As existing evaluation metrics, such as CLIPScore, fail to adequately capture the nuances of complex instructions, we introduce an evaluation toolkit that automates the quality assessment of generated images using a set of multi-dimensional metrics. The data and code are released at https://github.com/yczhou001/LongBench-T2I.