DiffDesign: Controllable Diffusion with Meta Prior for Efficient Interior Design Generation

TL;DR

DiffDesign introduces a controllable diffusion model with meta priors and a specialized dataset to generate interior designs efficiently, addressing the need for customizable and consistent outputs in interior design automation.

Contribution

The paper presents a novel controllable diffusion model with meta priors and a new interior design dataset, enabling efficient, customizable, and view-consistent interior design generation.

Findings

Effective control over design attributes like appearance, pose, and size.

High-quality, view-consistent interior design outputs.

Robust performance demonstrated on benchmark datasets.

Abstract

Interior design is a complex and creative discipline involving aesthetics, functionality, ergonomics, and materials science. Effective solutions must meet diverse requirements, typically producing multiple deliverables such as renderings and design drawings from various perspectives. Consequently, interior design processes are often inefficient and demand significant creativity. With advances in machine learning, generative models have emerged as a promising means of improving efficiency by creating designs from text descriptions or sketches. However, few generative works focus on interior design, leading to substantial discrepancies between outputs and practical needs, such as differences in size, spatial scope, and the lack of controllable generation quality. To address these challenges, we propose DiffDesign, a controllable diffusion model with meta priors for efficient interior design generation. Specifically, we utilize the generative priors of a 2D diffusion model pre-trained on a large image dataset as our rendering backbone. We further guide the denoising process by disentangling cross-attention control over design attributes, such as appearance, pose, and size, and introduce an optimal transfer-based alignment module to enforce view consistency. Simultaneously, we construct an interior design-specific dataset, DesignHelper, consisting of over 400 solutions across more than 15 spatial types and 15 design styles. This dataset helps fine-tune DiffDesign. Extensive experiments conducted on various benchmark datasets demonstrate the effectiveness and robustness of DiffDesign.