Variational Transformer Networks for Layout Generation

TL;DR

This paper introduces Variational Transformer Networks, a novel model that uses self-attention within a VAE framework to generate diverse, high-quality layouts for various design tasks without explicit supervision.

Contribution

The paper presents the first application of self-attention in a variational autoencoder for layout generation, achieving state-of-the-art diversity and quality.

Findings

High resemblance of generated layouts to training data

Achieves state-of-the-art diversity and perceptual quality

Effective in document layout detection pipeline

Abstract

Generative models able to synthesize layouts of different kinds (e.g. documents, user interfaces or furniture arrangements) are a useful tool to aid design processes and as a first step in the generation of synthetic data, among other tasks. We exploit the properties of self-attention layers to capture high level relationships between elements in a layout, and use these as the building blocks of the well-known Variational Autoencoder (VAE) formulation. Our proposed Variational Transformer Network (VTN) is capable of learning margins, alignments and other global design rules without explicit supervision. Layouts sampled from our model have a high degree of resemblance to the training data, while demonstrating appealing diversity. In an extensive evaluation on publicly available benchmarks for different layout types VTNs achieve state-of-the-art diversity and perceptual quality. Additionally, we show the capabilities of this method as part of a document layout detection pipeline.