Dependency-Aware Discrete Diffusion for Scene Graph Generation

TL;DR

This paper introduces a dependency-aware discrete diffusion model for scene graph generation from natural language, improving structural fidelity and downstream image composition.

Contribution

It proposes a hierarchically constrained diffusion approach that decouples structure and semantics, enabling better scene graph generation aligned with text.

Findings

Outperforms existing graph generation baselines on standard benchmarks.

Enhances compositional alignment in downstream image generation.

Captures hierarchical dependencies in scene graphs effectively.

Abstract

Scene graphs (SGs) represent objects and their relationships as structured graphs, enabling applications in image generation, robotics, and 3D understanding. Recent work suggests that conditioning image generation on scene graphs improves compositional fidelity compared to text-only prompting. However, since users typically provide text rather than structured graphs, a key challenge is to generate scene graphs from natural language. Prior work on discrete diffusion has demonstrated success in generating generic graphs such as molecules and circuits, but fails to account for the hierarchical structure and strong dependencies between objects, edges, and relations in scene graphs. We address this limitation by introducing a dependency-aware, hierarchically constrained discrete diffusion model for scene graph generation. Our approach decouples structure and semantics across the forward and reverse processes, enabling the model to capture conditional dependencies. At inference time, we perform training-free conditioning to sample text-aligned scene graphs. We evaluate our method on standard SG benchmarks and demonstrate improvements over both continuous and discrete graph generation baselines across graph and layout metrics. When fed to downstream image generation, our approach yields improved compositional alignment compared to text-to-image models, particularly in multi-object scenarios.