Mixed Diffusion for 3D Indoor Scene Synthesis

TL;DR

This paper introduces MiDiffusion, a novel mixed discrete-continuous diffusion model that synthesizes realistic 3D indoor scenes from floor plans, outperforming existing methods and enabling flexible scene completion.

Contribution

The paper presents MiDiffusion, the first diffusion model tailored for floor-conditioned 3D indoor scene synthesis with mixed discrete and continuous data handling.

Findings

Outperforms state-of-the-art autoregressive and diffusion models on 3D-FRONT dataset.

Effectively handles partial object constraints without task-specific training.

Demonstrates advantages in scene completion and furniture arrangement tasks.

Abstract

Generating realistic 3D scenes is an area of growing interest in computer vision and robotics. However, creating high-quality, diverse synthetic 3D content often requires expert intervention, making it costly and complex. Recently, efforts to automate this process with learning techniques, particularly diffusion models, have shown significant improvements in tasks like furniture rearrangement. However, applying diffusion models to floor-conditioned indoor scene synthesis remains under-explored. This task is especially challenging as it requires arranging objects in continuous space while selecting from discrete object categories, posing unique difficulties for conventional diffusion methods. To bridge this gap, we present MiDiffusion, a novel mixed discrete-continuous diffusion model designed to synthesize plausible 3D indoor scenes given a floor plan and pre-arranged objects. We represent a scene layout by a 2D floor plan and a set of objects, each defined by category, location, size, and orientation. Our approach uniquely applies structured corruption across mixed discrete semantic and continuous geometric domains, resulting in a better-conditioned problem for denoising. Evaluated on the 3D-FRONT dataset, MiDiffusion outperforms state-of-the-art autoregressive and diffusion models in floor-conditioned 3D scene synthesis. Additionally, it effectively handles partial object constraints via a corruption-and-masking strategy without task-specific training, demonstrating advantages in scene completion and furniture arrangement tasks.