GenScan: A Generative Method for Populating Parametric 3D Scan Datasets

TL;DR

GenScan is an automated generative system that creates diverse, textured 3D building scene variations from existing scans, aiding data augmentation for 3D deep learning applications.

Contribution

It introduces a novel hybrid neural network and parametrizer-based system for generating customizable 3D scene variations from real scans.

Findings

Enables automated generation of diverse 3D scene variations

Incorporates style transfer for realistic textures

Facilitates data augmentation for 3D deep learning

Abstract

The availability of rich 3D datasets corresponding to the geometrical complexity of the built environments is considered an ongoing challenge for 3D deep learning methodologies. To address this challenge, we introduce GenScan, a generative system that populates synthetic 3D scan datasets in a parametric fashion. The system takes an existing captured 3D scan as an input and outputs alternative variations of the building layout including walls, doors, and furniture with corresponding textures. GenScan is a fully automated system that can also be manually controlled by a user through an assigned user interface. Our proposed system utilizes a combination of a hybrid deep neural network and a parametrizer module to extract and transform elements of a given 3D scan. GenScan takes advantage of style transfer techniques to generate new textures for the generated scenes. We believe our system would facilitate data augmentation to expand the currently limited 3D geometry datasets commonly used in 3D computer vision, generative design, and general 3D deep learning tasks.