ControlGUI: Guiding Generative GUI Exploration through Perceptual Visual Flow
Aryan Garg, Yue Jiang, Antti Oulasvirta

TL;DR
ControlGUI introduces a diffusion-based method enabling designers to generate diverse interface sketches with minimal effort using prompts, wireframes, and visual flows, facilitating rapid exploration of design spaces.
Contribution
It presents a novel, flexible control approach for generative interface sketching, allowing multi-input specification and improved alignment with design inputs.
Findings
Enables rapid generation of diverse interface sketches with minimal input.
Allows flexible combination of prompts, wireframes, and visual flows for control.
Outperforms existing models in aligning with specified inputs.
Abstract
During the early stages of interface design, designers need to produce multiple sketches to explore a design space. Design tools often fail to support this critical stage, because they insist on specifying more details than necessary. Although recent advances in generative AI have raised hopes of solving this issue, in practice they fail because expressing loose ideas in a prompt is impractical. In this paper, we propose a diffusion-based approach to the low-effort generation of interface sketches. It breaks new ground by allowing flexible control of the generation process via three types of inputs: A) prompts, B) wireframes, and C) visual flows. The designer can provide any combination of these as input at any level of detail, and will get a diverse gallery of low-fidelity solutions in response. The unique benefit is that large design spaces can be explored rapidly with very little…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
