EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing

TL;DR

EditRoom introduces a unified framework that uses large language models and diffusion techniques to enable comprehensive, natural language-guided 3D scene layout editing, significantly improving flexibility and accuracy.

Contribution

The paper presents a novel approach combining LLMs and diffusion models for versatile 3D scene editing guided by natural language commands, supported by a large-scale dataset.

Findings

Outperforms baselines in accuracy and coherence

Supports six types of scene layout edits

Introduces a large-scale dataset for training and evaluation

Abstract

Given the steep learning curve of professional 3D software and the time-consuming process of managing large 3D assets, language-guided 3D scene editing has significant potential in fields such as virtual reality, augmented reality, and gaming. However, recent approaches to language-guided 3D scene editing either require manual interventions or focus only on appearance modifications without supporting comprehensive scene layout changes. In response, we propose EditRoom, a unified framework capable of executing a variety of layout edits through natural language commands, without requiring manual intervention. Specifically, EditRoom leverages Large Language Models (LLMs) for command planning and generates target scenes using a diffusion-based method, enabling six types of edits: rotate, translate, scale, replace, add, and remove. To address the lack of data for language-guided 3D scene editing, we have developed an automatic pipeline to augment existing 3D scene synthesis datasets and introduced EditRoom-DB, a large-scale dataset with 83k editing pairs, for training and evaluation. Our experiments demonstrate that our approach consistently outperforms other baselines across all metrics, indicating higher accuracy and coherence in language-guided scene layout editing.