DeBaRA: Denoising-Based 3D Room Arrangement Generation

TL;DR

DeBaRA is a novel score-based model for generating diverse and realistic 3D indoor scene layouts, emphasizing precise spatial placement of objects for applications like synthesis, completion, and re-arrangement.

Contribution

We introduce DeBaRA, a lightweight, 3D spatially-aware score-based model that improves scene arrangement generation and offers versatile downstream applications.

Findings

Significant improvement over state-of-the-art methods.

Effective for scene synthesis, completion, and re-arrangement.

Introduces a novel Self Score Evaluation procedure.

Abstract

Generating realistic and diverse layouts of furnished indoor 3D scenes unlocks multiple interactive applications impacting a wide range of industries. The inherent complexity of object interactions, the limited amount of available data and the requirement to fulfill spatial constraints all make generative modeling for 3D scene synthesis and arrangement challenging. Current methods address these challenges autoregressively or by using off-the-shelf diffusion objectives by simultaneously predicting all attributes without 3D reasoning considerations. In this paper, we introduce DeBaRA, a score-based model specifically tailored for precise, controllable and flexible arrangement generation in a bounded environment. We argue that the most critical component of a scene synthesis system is to accurately establish the size and position of various objects within a restricted area. Based on this insight, we propose a lightweight conditional score-based model designed with 3D spatial awareness at its core. We demonstrate that by focusing on spatial attributes of objects, a single trained DeBaRA model can be leveraged at test time to perform several downstream applications such as scene synthesis, completion and re-arrangement. Further, we introduce a novel Self Score Evaluation procedure so it can be optimally employed alongside external LLM models. We evaluate our approach through extensive experiments and demonstrate significant improvement upon state-of-the-art approaches in a range of scenarios.