ToMigo: Interpretable Design Concept Graphs for Aligning Generative AI with Creative Intent

TL;DR

ToMigo introduces interpretable design concept graphs to improve alignment of generative AI with user creative intent, enabling better understanding, control, and customization of AI-generated designs.

Contribution

The paper presents ToMigo, a novel approach using design concept graphs to interpret and influence AI's understanding of user intent in graphic design.

Findings

High user alignment ratings with ToMigo

Users found interactive features useful for design control

Effective inference of user intent from images and text

Abstract

Generative AI often produces results misaligned with user intentions, for example, resolving ambiguous prompts in unexpected ways. Despite existing approaches to clarify intent, a major challenge remains: understanding and influencing AI's interpretation of user intent through simple, direct inputs requiring no expertise or rigid procedures. We present ToMigo, representing intent as design concept graphs: nodes represent choices of purpose, content, or style, while edges link them with interpretable explanations. Applied to graphic design, ToMigo infers intent from reference images and text. We derived a schema of node types and edges from pre-study data, informing a multimodal large language model to generate graphs aligning nodes externally with user intent and internally toward a unified design goal. This structure enables users to explore AI reasoning and directly manipulate the design concept. In our user studies, ToMigo received high alignment ratings and captured most user intentions well. Users reported greater control and found interactive features-editable graphs, reflective chats, concept-design realignment-useful for evolving and realizing their design ideas.