VASCAR: Content-Aware Layout Generation via Visual-Aware Self-Correction

TL;DR

VASCAR introduces a training-free, visual-aware self-correction method for layout generation that leverages large vision-language models to iteratively refine outputs based on visual feedback, achieving state-of-the-art results.

Contribution

The paper presents VASCAR, a novel approach enabling LVLMs to improve layout generation through visual-aware self-correction without additional training.

Findings

VASCAR achieves state-of-the-art layout quality.

It effectively refines outputs using visual feedback.

Demonstrates versatility across different LVLMs.

Abstract

Large language models (LLMs) have proven effective for layout generation due to their ability to produce structure-description languages, such as HTML or JSON. In this paper, we argue that while LLMs can perform reasonably well in certain cases, their intrinsic limitation of not being able to perceive images restricts their effectiveness in tasks requiring visual content, e.g., content-aware layout generation. Therefore, we explore whether large vision-language models (LVLMs) can be applied to content-aware layout generation. To this end, inspired by the iterative revision and heuristic evaluation workflow of designers, we propose the training-free Visual-Aware Self-Correction LAyout GeneRation (VASCAR). VASCAR enables LVLMs (e.g., GPT-4o and Gemini) iteratively refine their outputs with reference to rendered layout images, which are visualized as colored bounding boxes on poster background (i.e., canvas). Extensive experiments and user study demonstrate VASCAR's effectiveness, achieving state-of-the-art (SOTA) layout generation quality. Furthermore, the generalizability of VASCAR across GPT-4o and Gemini demonstrates its versatility.