ReLayout: Versatile and Structure-Preserving Design Layout Editing via Relation-Aware Design Reconstruction

TL;DR

ReLayout is a novel framework that enables versatile, structure-preserving design layout editing driven by relation-aware reconstruction and large language models, overcoming data scarcity and ambiguity in user intents.

Contribution

The paper introduces ReLayout, a new method for design layout editing that does not require triplet training data and preserves layout structure using relation graphs and self-supervised learning.

Findings

ReLayout outperforms baselines in editing quality and accuracy.

It effectively preserves layout structure during editing.

The approach is versatile across different editing actions.

Abstract

Automated redesign without manual adjustments marks a key step forward in the design workflow. In this work, we focus on a foundational redesign task termed design layout editing, which seeks to autonomously modify the geometric composition of a design based on user intents. To overcome the ambiguity of user needs expressed in natural language, we introduce four basic and important editing actions and standardize the format of editing operations. The underexplored task presents a unique challenge: satisfying specified editing operations while simultaneously preserving the layout structure of unedited elements. Besides, the scarcity of triplet (original design, editing operation, edited design) samples poses another formidable challenge. To this end, we present ReLayout, a novel framework for versatile and structure-preserving design layout editing that operates without triplet data. Specifically, ReLayout first introduces the relation graph, which contains the position and size relationships among unedited elements, as the constraint for layout structure preservation. Then, relation-aware design reconstruction (RADR) is proposed to bypass the data challenge. By learning to reconstruct a design from its elements, a relation graph, and a synthesized editing operation, RADR effectively emulates the editing process in a self-supervised manner. A multi-modal large language model serves as the backbone for RADR, unifying multiple editing actions within a single model and thus achieving versatile editing after fine-tuning. Qualitative, quantitative results and user studies show that ReLayout significantly outperforms the baseline models in terms of editing quality, accuracy, and layout structure preservation.