StructLayoutFormer:Conditional Structured Layout Generation via Structure Serialization and Disentanglement

TL;DR

StructLayoutFormer is a Transformer-based model that generates structured 2D layouts conditionally, explicitly modeling layout structures through serialization and disentanglement, outperforming existing methods in structure control and realism.

Contribution

It introduces a novel serialization and disentanglement scheme enabling conditional structured layout generation with explicit structural control.

Findings

Outperforms existing data-driven layout generation methods.

Effectively extracts and transfers layout structures.

Produces realistic and controllable structured layouts.

Abstract

Structured layouts are preferable in many 2D visual contents (\eg, GUIs, webpages) since the structural information allows convenient layout editing. Computational frameworks can help create structured layouts but require heavy labor input. Existing data-driven approaches are effective in automatically generating fixed layouts but fail to produce layout structures. We present StructLayoutFormer, a novel Transformer-based approach for conditional structured layout generation. We use a structure serialization scheme to represent structured layouts as sequences. To better control the structures of generated layouts, we disentangle the structural information from the element placements. Our approach is the first data-driven approach that achieves conditional structured layout generation and produces realistic layout structures explicitly. We compare our approach with existing data-driven layout generation approaches by including post-processing for structure extraction. Extensive experiments have shown that our approach exceeds these baselines in conditional structured layout generation. We also demonstrate that our approach is effective in extracting and transferring layout structures. The code is publicly available at %\href{https://github.com/Teagrus/StructLayoutFormer} {https://github.com/Teagrus/StructLayoutFormer}.