Graph Transformer GANs for Graph-Constrained House Generation

TL;DR

This paper introduces GTGAN, a graph Transformer GAN that effectively models local and global node relations for graph-constrained house generation, achieving state-of-the-art results in layout and roof synthesis.

Contribution

The paper proposes a novel graph Transformer generator with connected and non-connected node attention, a graph modeling block, a node classification discriminator, and a cycle-consistency loss for improved house graph generation.

Findings

Achieves state-of-the-art results on house layout and roof generation tasks.

Demonstrates effectiveness through quantitative scores and visual realism.

Outperforms previous methods by large margins.

Abstract

We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations in an end-to-end fashion for the challenging graph-constrained house generation task. The proposed graph-Transformer-based generator includes a novel graph Transformer encoder that combines graph convolutions and self-attentions in a Transformer to model both local and global interactions across connected and non-connected graph nodes. Specifically, the proposed connected node attention (CNA) and non-connected node attention (NNA) aim to capture the global relations across connected nodes and non-connected nodes in the input graph, respectively. The proposed graph modeling block (GMB) aims to exploit local vertex interactions based on a house layout topology. Moreover, we propose a new node classification-based discriminator to preserve the high-level semantic and discriminative node features for different house components. Finally, we propose a novel graph-based cycle-consistency loss that aims at maintaining the relative spatial relationships between ground truth and predicted graphs. Experiments on two challenging graph-constrained house generation tasks (i.e., house layout and roof generation) with two public datasets demonstrate the effectiveness of GTGAN in terms of objective quantitative scores and subjective visual realism. New state-of-the-art results are established by large margins on both tasks.