SimGraph: A Unified Framework for Scene Graph-Based Image Generation and Editing

TL;DR

SimGraph is a unified framework that leverages scene graphs to improve control, consistency, and quality in both image generation and editing tasks within generative AI.

Contribution

It introduces a novel integrated scene graph-based model combining token and diffusion methods for enhanced image synthesis and editing.

Findings

Outperforms existing state-of-the-art methods in quality and consistency

Provides precise control over object interactions and spatial arrangements

Ensures high-quality, coherent image generation and editing

Abstract

Recent advancements in Generative Artificial Intelligence (GenAI) have significantly enhanced the capabilities of both image generation and editing. However, current approaches often treat these tasks separately, leading to inefficiencies and challenges in maintaining spatial consistency and semantic coherence between generated content and edits. Moreover, a major obstacle is the lack of structured control over object relationships and spatial arrangements. Scene graph-based methods, which represent objects and their interrelationships in a structured format, offer a solution by providing greater control over composition and interactions in both image generation and editing. To address this, we introduce SimGraph, a unified framework that integrates scene graph-based image generation and editing, enabling precise control over object interactions, layouts, and spatial coherence. In particular, our framework integrates token-based generation and diffusion-based editing within a single scene graph-driven model, ensuring high-quality and consistent results. Through extensive experiments, we empirically demonstrate that our approach outperforms existing state-of-the-art methods.