PromptCrafter: Crafting Text-to-Image Prompt through Mixed-Initiative Dialogue with LLM

TL;DR

PromptCrafter is a mixed-initiative system that enables users to iteratively craft and refine text-to-image prompts through dialogue with a large language model, enhancing exploration and clarity of intent.

Contribution

It introduces a novel interactive system for step-by-step prompt creation and refinement in text-to-image generation using LLMs, supporting user exploration and prompt clarification.

Findings

Supports efficient exploration of model capabilities

Enables iterative prompt refinement through dialogue

Allows users to review and revert prompt steps

Abstract

Text-to-image generation model is able to generate images across a diverse range of subjects and styles based on a single prompt. Recent works have proposed a variety of interaction methods that help users understand the capabilities of models and utilize them. However, how to support users to efficiently explore the model's capability and to create effective prompts are still open-ended research questions. In this paper, we present PromptCrafter, a novel mixed-initiative system that allows step-by-step crafting of text-to-image prompt. Through the iterative process, users can efficiently explore the model's capability, and clarify their intent. PromptCrafter also supports users to refine prompts by answering various responses to clarifying questions generated by a Large Language Model. Lastly, users can revert to a desired step by reviewing the work history. In this workshop paper, we discuss the design process of PromptCrafter and our plans for follow-up studies.