Frontend Diffusion: Exploring Intent-Based User Interfaces through Abstract-to-Detailed Task Transitions

TL;DR

This paper introduces Frontend Diffusion, an AI-powered tool that translates user sketches into complete websites through a three-stage process, advancing intent-based user interfaces by reducing manual effort.

Contribution

It presents a novel end-to-end system leveraging large language models for abstract-to-detailed task transitions in frontend development, bridging user intent and implementation.

Findings

Demonstrates high-quality website generation from sketches.

Reduces human intervention in complex frontend tasks.

Opens pathways for applying task transition approaches to other domains.

Abstract

The emergence of Generative AI is catalyzing a paradigm shift in user interfaces from command-based to intent-based outcome specification. In this paper, we explore abstract-to-detailed task transitions in the context of frontend code generation as a step towards intent-based user interfaces, aiming to bridge the gap between abstract user intentions and concrete implementations. We introduce Frontend Diffusion, an end-to-end LLM-powered tool that generates high-quality websites from user sketches. The system employs a three-stage task transition process: sketching, writing, and coding. We demonstrate the potential of task transitions to reduce human intervention and communication costs in complex tasks. Our work also opens avenues for exploring similar approaches in other domains, potentially extending to more complex, interdependent tasks such as video production.