What Makes a Scene ? Scene Graph-based Evaluation and Feedback for Controllable Generation
Zuyao Chen, Jinlin Wu, Zhen Lei, Chang Wen Chen

TL;DR
This paper introduces Scene-Bench, a new benchmark with a large dataset and a novel evaluation metric, SGScore, to improve factual accuracy in scene graph-based image generation, along with a feedback pipeline for iterative refinement.
Contribution
The paper presents Scene-Bench, a large-scale dataset, SGScore for better evaluation, and a feedback method to enhance controllable scene image generation.
Findings
Scene-Bench outperforms existing benchmarks in evaluating complex scenes.
SGScore provides more accurate assessment of object and relationship accuracy.
Feedback pipeline improves factual consistency of generated images.
Abstract
While text-to-image generation has been extensively studied, generating images from scene graphs remains relatively underexplored, primarily due to challenges in accurately modeling spatial relationships and object interactions. To fill this gap, we introduce Scene-Bench, a comprehensive benchmark designed to evaluate and enhance the factual consistency in generating natural scenes. Scene-Bench comprises MegaSG, a large-scale dataset of one million images annotated with scene graphs, facilitating the training and fair comparison of models across diverse and complex scenes. Additionally, we propose SGScore, a novel evaluation metric that leverages chain-of-thought reasoning capabilities of multimodal large language models (LLMs) to assess both object presence and relationship accuracy, offering a more effective measure of factual consistency than traditional metrics like FID and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsData Visualization and Analytics · Artificial Intelligence in Games
