3D Scene Diffusion Guidance using Scene Graphs

TL;DR

This paper introduces a novel method for guiding 3D scene diffusion generation using scene graphs, which effectively incorporate complex spatial relationships to improve scene alignment.

Contribution

The paper presents a new approach that integrates scene graphs into 3D diffusion models, enhancing spatial relationship modeling over traditional text-based guidance.

Findings

Improved alignment between scene descriptions and generated scenes.

Use of relational graph convolutional blocks enhances spatial relationship encoding.

Significant performance gains demonstrated in 3D scene synthesis.

Abstract

Guided synthesis of high-quality 3D scenes is a challenging task. Diffusion models have shown promise in generating diverse data, including 3D scenes. However, current methods rely directly on text embeddings for controlling the generation, limiting the incorporation of complex spatial relationships between objects. We propose a novel approach for 3D scene diffusion guidance using scene graphs. To leverage the relative spatial information the scene graphs provide, we make use of relational graph convolutional blocks within our denoising network. We show that our approach significantly improves the alignment between scene description and generated scene.