Uni-Layout: Integrating Human Feedback in Unified Layout Generation and Evaluation

TL;DR

Uni-Layout introduces a unified framework for layout generation and evaluation that incorporates human feedback, enabling more versatile and perceptually aligned layout design through a large-scale annotated dataset and a human-mimicking evaluator.

Contribution

The paper presents Uni-Layout, a novel system integrating layout generation and human-aligned evaluation, supported by a large human feedback dataset and a dynamic optimization method.

Findings

Outperforms task-specific and general-purpose methods

Effectively aligns generator and evaluator with human judgments

Demonstrates significant improvements in layout quality and evaluation accuracy

Abstract

Layout generation plays a crucial role in enhancing both user experience and design efficiency. However, current approaches suffer from task-specific generation capabilities and perceptually misaligned evaluation metrics, leading to limited applicability and ineffective measurement. In this paper, we propose \textit{Uni-Layout}, a novel framework that achieves unified generation, human-mimicking evaluation and alignment between the two. For universal generation, we incorporate various layout tasks into a single taxonomy and develop a unified generator that handles background or element contents constrained tasks via natural language prompts. To introduce human feedback for the effective evaluation of layouts, we build \textit{Layout-HF100k}, the first large-scale human feedback dataset with 100,000 expertly annotated layouts. Based on \textit{Layout-HF100k}, we introduce a human-mimicking evaluator that integrates visual and geometric information, employing a Chain-of-Thought mechanism to conduct qualitative assessments alongside a confidence estimation module to yield quantitative measurements. For better alignment between the generator and the evaluator, we integrate them into a cohesive system by adopting Dynamic-Margin Preference Optimization (DMPO), which dynamically adjusts margins based on preference strength to better align with human judgments. Extensive experiments show that \textit{Uni-Layout} significantly outperforms both task-specific and general-purpose methods. Our code is publicly available at https://github.com/JD-GenX/Uni-Layout.