VisionDirector: Vision-Language Guided Closed-Loop Refinement for Generative Image Synthesis

TL;DR

VisionDirector is a novel vision-language guided system that improves generative image synthesis by effectively handling complex, multi-goal prompts through structured goal extraction, staged editing, and semantic verification, leading to state-of-the-art results.

Contribution

The paper introduces VisionDirector, a training-free, goal-oriented supervision method that enhances multi-goal image editing and synthesis, outperforming existing models on complex benchmarks.

Findings

Achieves 7% improvement on GenEval

Reduces edit steps from 4.2 to 3.1

Enhances consistency in typography and pose editing

Abstract

Generative models can now produce photorealistic imagery, yet they still struggle with the long, multi-goal prompts that professional designers issue. To expose this gap and better evaluate models' performance in real-world settings, we introduce Long Goal Bench (LGBench), a 2,000-task suite (1,000 T2I and 1,000 I2I) whose average instruction contains 18 to 22 tightly coupled goals spanning global layout, local object placement, typography, and logo fidelity. We find that even state-of-the-art models satisfy fewer than 72 percent of the goals and routinely miss localized edits, confirming the brittleness of current pipelines. To address this, we present VisionDirector, a training-free vision-language supervisor that (i) extracts structured goals from long instructions, (ii) dynamically decides between one-shot generation and staged edits, (iii) runs micro-grid sampling with semantic verification and rollback after every edit, and (iv) logs goal-level rewards. We further fine-tune the planner with Group Relative Policy Optimization, yielding shorter edit trajectories (3.1 versus 4.2 steps) and stronger alignment. VisionDirector achieves new state of the art on GenEval (plus 7 percent overall) and ImgEdit (plus 0.07 absolute) while producing consistent qualitative improvements on typography, multi-object scenes, and pose editing.