LAION-SG: An Enhanced Large-Scale Dataset for Training Complex Image-Text Models with Structural Annotations

TL;DR

This paper introduces LAION-SG, a large-scale dataset with detailed scene graph annotations for improved complex image generation, and demonstrates its effectiveness through a new foundation model and benchmark.

Contribution

The paper presents LAION-SG, a novel dataset with structural annotations, and a foundation model trained on it, advancing compositional image generation capabilities.

Findings

Models trained on LAION-SG outperform existing datasets in complex scene generation.

LAION-SG enables better understanding of object relationships in images.

The new benchmark sets a standard for evaluating compositional image generation.

Abstract

Recent advances in text-to-image (T2I) generation have shown remarkable success in producing high-quality images from text. However, existing T2I models show decayed performance in compositional image generation involving multiple objects and intricate relationships. We attribute this problem to limitations in existing datasets of image-text pairs, which lack precise inter-object relationship annotations with prompts only. To address this problem, we construct LAION-SG, a large-scale dataset with high-quality structural annotations of scene graphs (SG), which precisely describe attributes and relationships of multiple objects, effectively representing the semantic structure in complex scenes. Based on LAION-SG, we train a new foundation model SDXL-SG to incorporate structural annotation information into the generation process. Extensive experiments show advanced models trained on our LAION-SG boast significant performance improvements in complex scene generation over models on existing datasets. We also introduce CompSG-Bench, a benchmark that evaluates models on compositional image generation, establishing a new standard for this domain. Our annotations with the associated processing code, the foundation model and the benchmark protocol are publicly available at https://github.com/mengcye/LAION-SG.