Synthesizing 3D Abstractions by Inverting Procedural Buildings with Transformers

TL;DR

This paper presents a method using transformers to invert procedural building models from point clouds, enabling the generation of accurate, structured 3D abstractions that are efficient and regular.

Contribution

It introduces a novel transformer-based approach to invert procedural building models from point clouds, creating structured abstractions with strong priors for regularity.

Findings

Achieves high reconstruction accuracy in geometry and structure.

Produces structurally consistent inpainting.

Leverages expressive procedural models for efficient rendering.

Abstract

We generate abstractions of buildings, reflecting the essential aspects of their geometry and structure, by learning to invert procedural models. We first build a dataset of abstract procedural building models paired with simulated point clouds and then learn the inverse mapping through a transformer. Given a point cloud, the trained transformer then infers the corresponding abstracted building in terms of a programmatic language description. This approach leverages expressive procedural models developed for gaming and animation, and thereby retains desirable properties such as efficient rendering of the inferred abstractions and strong priors for regularity and symmetry. Our approach achieves good reconstruction accuracy in terms of geometry and structure, as well as structurally consistent inpainting.