BuilDiff: 3D Building Shape Generation using Single-Image Conditional Point Cloud Diffusion Models

TL;DR

This paper introduces BuilDiff, a flexible 3D building shape generation method using point cloud diffusion models conditioned on single images, capable of handling general-view images with improved quality.

Contribution

The paper presents a novel point cloud diffusion approach with image conditioning and regularization, enabling scalable and high-quality 3D building generation from single images.

Findings

Outperforms previous methods in building generation quality

Effective in synthesizing building roofs and structures

Validated on two new datasets with extensive experiments

Abstract

3D building generation with low data acquisition costs, such as single image-to-3D, becomes increasingly important. However, most of the existing single image-to-3D building creation works are restricted to those images with specific viewing angles, hence they are difficult to scale to general-view images that commonly appear in practical cases. To fill this gap, we propose a novel 3D building shape generation method exploiting point cloud diffusion models with image conditioning schemes, which demonstrates flexibility to the input images. By cooperating two conditional diffusion models and introducing a regularization strategy during denoising process, our method is able to synthesize building roofs while maintaining the overall structures. We validate our framework on two newly built datasets and extensive experiments show that our method outperforms previous works in terms of building generation quality.