Generating 3D House Wireframes with Semantics

TL;DR

This paper introduces a novel autoregressive approach for generating semantically enriched 3D house wireframes, improving coherence and semantic integration over traditional methods.

Contribution

It proposes a unified wire-based representation and a two-phase model combining a graph autoencoder with a transformer decoder for semantic-aware 3D wireframe generation.

Findings

Outperforms existing models in accuracy and semantic fidelity

Produces detailed, component-segmented wireframes

Validates effectiveness on a comprehensive house dataset

Abstract

We present a new approach for generating 3D house wireframes with semantic enrichment using an autoregressive model. Unlike conventional generative models that independently process vertices, edges, and faces, our approach employs a unified wire-based representation for improved coherence in learning 3D wireframe structures. By re-ordering wire sequences based on semantic meanings, we facilitate seamless semantic integration during sequence generation. Our two-phase technique merges a graph-based autoencoder with a transformer-based decoder to learn latent geometric tokens and generate semantic-aware wireframes. Through iterative prediction and decoding during inference, our model produces detailed wireframes that can be easily segmented into distinct components, such as walls, roofs, and rooms, reflecting the semantic essence of the shape. Empirical results on a comprehensive house dataset validate the superior accuracy, novelty, and semantic fidelity of our model compared to existing generative models. More results and details can be found on https://vcc.tech/research/2024/3DWire.