Taming Text-to-Image Synthesis for Novices: User-centric Prompt Generation via Multi-turn Guidance

TL;DR

This paper introduces DialPrompt, a dialogue-based prompt generation system for text-to-image synthesis that enhances user control and understanding, especially for novices, by guiding multi-turn interactions based on essential prompt dimensions.

Contribution

It presents a novel multi-turn, user-centric prompt generation framework for TIS, curated from expert insights, improving novice user experience and interpretability.

Findings

Significant improvement in user-centricity scores.

Maintains high-quality image synthesis results.

Highly rated by novice users in evaluations.

Abstract

The emergence of text-to-image synthesis (TIS) models has significantly influenced digital image creation by producing high-quality visuals from written descriptions. Yet these models are sensitive on textual prompts, posing a challenge for novice users who may not be familiar with TIS prompt writing. Existing solutions relieve this via automatic prompt expansion or generation from a user query. However, this single-turn manner suffers from limited user-centricity in terms of result interpretability and user interactivity. Thus, we propose DialPrompt, a dialogue-based TIS prompt generation model that emphasizes user experience for novice users. DialPrompt is designed to follow a multi-turn workflow, where in each round of dialogue the model guides user to express their preferences on possible optimization dimensions before generating the final TIS prompt. To achieve this, we mined 15 essential dimensions for high-quality prompts from advanced users and curated a multi-turn dataset. Through training on this dataset, DialPrompt improves user-centricity by allowing users to perceive and control the creation process of TIS prompts. Experiments indicate that DialPrompt improves significantly in user-centricity score compared with existing approaches while maintaining a competitive quality of synthesized images. In our user evaluation, DialPrompt is highly rated by 19 human reviewers (especially novices).