A Parse-Then-Place Approach for Generating Graphic Layouts from Textual Descriptions

TL;DR

This paper introduces a parse-then-place method for generating graphic layouts from text descriptions, using an intermediate representation and Transformer models to handle implicit and incomplete constraints, improving layout generation quality.

Contribution

The work presents a novel two-stage parse-then-place approach with an intermediate representation, enabling effective Text-to-Layout generation considering complex constraints.

Findings

Outperforms baseline models in layout accuracy

Effective handling of implicit and incomplete constraints

Validated by user studies and quantitative metrics

Abstract

Creating layouts is a fundamental step in graphic design. In this work, we propose to use text as the guidance to create graphic layouts, i.e., Text-to-Layout, aiming to lower the design barriers. Text-to-Layout is a challenging task, because it needs to consider the implicit, combined, and incomplete layout constraints from text, each of which has not been studied in previous work. To address this, we present a two-stage approach, named parse-then-place. The approach introduces an intermediate representation (IR) between text and layout to represent diverse layout constraints. With IR, Text-to-Layout is decomposed into a parse stage and a place stage. The parse stage takes a textual description as input and generates an IR, in which the implicit constraints from the text are transformed into explicit ones. The place stage generates layouts based on the IR. To model combined and incomplete constraints, we use a Transformer-based layout generation model and carefully design a way to represent constraints and layouts as sequences. Besides, we adopt the pretrain-then-finetune strategy to boost the performance of the layout generation model with large-scale unlabeled layouts. To evaluate our approach, we construct two Text-to-Layout datasets and conduct experiments on them. Quantitative results, qualitative analysis, and user studies demonstrate the effectiveness of our approach.