DraCo: Draft as CoT for Text-to-Image Preview and Rare Concept Generation

TL;DR

DraCo introduces a novel draft-as-CoT approach that uses a low-resolution image preview and verification to improve text-to-image generation, especially for rare concepts, by combining textual and visual reasoning.

Contribution

The paper proposes DraCo, a new interleaved reasoning paradigm that integrates visual drafts with textual planning, enhancing the generation of complex and rare concepts in text-to-image tasks.

Findings

Achieves +8% on GenEval and +0.91 on Imagine-Bench benchmarks.

Significantly outperforms direct generation and other CoT-based methods.

Supports training with the curated DraCo-240K dataset.

Abstract

Recent unified multimodal large language models (MLLMs) have shown impressive capabilities, incorporating chain-of-thought (CoT) reasoning for enhanced text-to-image generation. However, existing approaches remain limited, either treating the model merely as a standalone generator or relying on abstract textual planning. To this end, we propose Draft-as-CoT (DraCo), a novel interleaved reasoning paradigm that fully leverages both textual and visual contents in CoT for better planning and verification. Our method first generates a low-resolution draft image as preview, providing more concrete and structural visual planning and guidance. Then, we employ the model's inherent understanding capability to verify potential semantic misalignments between the draft and input prompt, and performs refinement through selective corrections with super-resolution. In this way, our approach addresses two fundamental challenges: the coarse-grained nature of textual planning and the difficulty in generating rare attribute combinations. To support training, we curate DraCo-240K, aiming to enhance three atomic capabilities spanning general correction, instance manipulation, and layout reorganization. Supported by DraCo-CFG, a specialized classifier-free guidance (CFG) strategy for interleaved reasoning, DraCo achieves a tremendous increase on GenEval (+8%), Imagine-Bench (+0.91), and GenEval++ (+3%), significantly outperforming direct generation and other generation methods empowered by CoT.