CrowdGenUI: Aligning LLM-Based UI Generation with Crowdsourced User Preferences

TL;DR

CrowdGenUI enhances LLM-based UI generation by incorporating crowdsourced user preferences, resulting in more personalized and task-aligned interfaces, as validated through a user study in the image editing domain.

Contribution

This paper introduces CrowdGenUI, a novel framework that integrates real user preferences into LLM-driven UI generation to improve personalization and task relevance.

Findings

Generated UIs better match user intentions

Preference-guided UI generation improves relevance

Framework effectively incorporates user preferences

Abstract

Large Language Models (LLMs) have demonstrated remarkable potential across various design domains, including user interface (UI) generation. However, current LLMs for UI generation tend to offer generic solutions that lack a nuanced understanding of task context and user preferences. We present CrowdGenUI, a framework that enhances LLM-based UI generation with crowdsourced user preferences. This framework addresses the limitations by guiding LLM reasoning with real user preferences, enabling the generation of UI widgets that reflect user needs and task-specific requirements. We evaluate our framework in the image editing domain by collecting a library of 720 user preferences from 50 participants, covering preferences such as predictability, efficiency, and explorability of various UI widgets. A user study (N=78) demonstrates that UIs generated with our preference-guided framework can better match user intentions compared to those generated by LLMs alone, highlighting the effectiveness of our proposed framework. We further discuss the study findings and present insights for future research on LLM-based user-centered UI generation.