LayoutRAG: Retrieval-Augmented Model for Content-agnostic Conditional Layout Generation

TL;DR

LayoutRAG introduces a retrieval-augmented approach for content-agnostic conditional layout generation, leveraging reference layouts to guide the process and improve the quality of generated visual arrangements.

Contribution

The paper presents a novel retrieval-based method with condition-modulated attention for more effective and accurate layout generation under specified constraints.

Findings

Outperforms existing state-of-the-art models in layout quality.

Successfully generates layouts meeting given conditions.

Demonstrates the effectiveness of retrieval-guided guidance in layout generation.

Abstract

Controllable layout generation aims to create plausible visual arrangements of element bounding boxes within a graphic design according to certain optional constraints, such as the type or position of a specific component. While recent diffusion or flow-matching models have achieved considerable advances in multifarious conditional generation tasks, there remains considerable room for generating optimal arrangements under given conditions. In this work, we propose to carry out layout generation through retrieving by conditions and reference-guided generation. Specifically, we retrieve appropriate layout templates according to given conditions as references. The references are then utilized to guide the denoising or flow-based transport process. By retrieving layouts compatible with the given conditions, we can uncover the potential information not explicitly provided in the given condition. Such an approach offers more effective guidance to the model during the generation process, in contrast to previous models that feed the condition to the model and let the model infer the unprovided layout attributes directly. Meanwhile, we design a condition-modulated attention that selectively absorbs retrieval knowledge, adapting to the difference between retrieved templates and given conditions. Extensive experiment results show that our method successfully produces high-quality layouts that meet the given conditions and outperforms existing state-of-the-art models. Code will be released upon acceptance.