APD-Agents: A Large Language Model-Driven Multi-Agents Collaborative Framework for Automated Page Design

TL;DR

This paper introduces APD-agents, a multi-agent framework driven by large language models to automate mobile app page design, reducing manual effort and improving consistency through collaborative agent interactions.

Contribution

The paper presents a novel multi-agent system utilizing LLMs for automated mobile app page layout design, integrating multiple specialized agents for content parsing, layout generation, and recursive component creation.

Findings

Achieves state-of-the-art performance on RICO dataset

Demonstrates effective collaboration among agents for design tasks

Reduces manual effort in mobile app page layout creation

Abstract

Layout design is a crucial step in developing mobile app pages. However, crafting satisfactory designs is time-intensive for designers: they need to consider which controls and content to present on the page, and then repeatedly adjust their size, position, and style for better aesthetics and structure. Although many design software can now help to perform these repetitive tasks, extensive training is needed to use them effectively. Moreover, collaborative design across app pages demands extra time to align standards and ensure consistent styling. In this work, we propose APD-agents, a large language model (LLM) driven multi-agent framework for automated page design in mobile applications. Our framework contains OrchestratorAgent, SemanticParserAgent, PrimaryLayoutAgent, TemplateRetrievalAgent, and RecursiveComponentAgent. Upon receiving the user's description of the page, the OrchestratorAgent can dynamically can direct other agents to accomplish users' design task. To be specific, the SemanticParserAgent is responsible for converting users' descriptions of page content into structured data. The PrimaryLayoutAgent can generate an initial coarse-grained layout of this page. The TemplateRetrievalAgent can fetch semantically relevant few-shot examples and enhance the quality of layout generation. Besides, a RecursiveComponentAgent can be used to decide how to recursively generate all the fine-grained sub-elements it contains for each element in the layout. Our work fully leverages the automatic collaboration capabilities of large-model-driven multi-agent systems. Experimental results on the RICO dataset show that our APD-agents achieve state-of-the-art performance.