Semi-Supervised Adversarial Recognition of Refined Window Structures for Inverse Procedural Fa\c{c}ade Modeling

TL;DR

This paper introduces a semi-supervised adversarial approach for recognizing and modeling window structures in fa extbackslash c extbackslash ade images, reducing the need for extensive labeled data and improving accuracy.

Contribution

It presents a novel semi-supervised adversarial training method integrated into inverse procedural modeling for fa extbackslash c extbackslash ade, enhancing recognition accuracy with limited labeled samples.

Findings

10% improvement in classification accuracy

50% improvement in parameter estimation

Better performance on unseen fa extbackslash c extbackslash ade styles

Abstract

Deep learning methods are notoriously data-hungry, which requires a large number of labeled samples. Unfortunately, the large amount of interactive sample labeling efforts has dramatically hindered the application of deep learning methods, especially for 3D modeling tasks, which require heterogeneous samples. To alleviate the work of data annotation for learned 3D modeling of fa\c{c}ades, this paper proposed a semi-supervised adversarial recognition strategy embedded in inverse procedural modeling. Beginning with textured LOD-2 (Level-of-Details) models, we use the classical convolutional neural networks to recognize the types and estimate the parameters of windows from image patches. The window types and parameters are then assembled into procedural grammar. A simple procedural engine is built inside an existing 3D modeling software, producing fine-grained window geometries. To obtain a useful model from a few labeled samples, we leverage the generative adversarial network to train the feature extractor in a semi-supervised manner. The adversarial training strategy can also exploit unlabeled data to make the training phase more stable. Experiments using publicly available fa\c{c}ade image datasets reveal that the proposed training strategy can obtain about 10% improvement in classification accuracy and 50% improvement in parameter estimation under the same network structure. In addition, performance gains are more pronounced when testing against unseen data featuring different fa\c{c}ade styles.