LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators

TL;DR

LayoutGAN is a novel GAN architecture that generates realistic graphic layouts by modeling geometric relations and using a wireframe discriminator to improve alignment and structure.

Contribution

The paper introduces LayoutGAN, which incorporates a differentiable wireframe rendering layer and self-attention to synthesize and refine graphic layouts more effectively than previous methods.

Findings

Effective in MNIST digit layout generation

Produces realistic document and scene layouts

Improves alignment accuracy in generated layouts

Abstract

Layout is important for graphic design and scene generation. We propose a novel Generative Adversarial Network, called LayoutGAN, that synthesizes layouts by modeling geometric relations of different types of 2D elements. The generator of LayoutGAN takes as input a set of randomly-placed 2D graphic elements and uses self-attention modules to refine their labels and geometric parameters jointly to produce a realistic layout. Accurate alignment is critical for good layouts. We thus propose a novel differentiable wireframe rendering layer that maps the generated layout to a wireframe image, upon which a CNN-based discriminator is used to optimize the layouts in image space. We validate the effectiveness of LayoutGAN in various experiments including MNIST digit generation, document layout generation, clipart abstract scene generation and tangram graphic design.