How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Models

TL;DR

This paper explores the potential and challenges of using prompt-based zero- and few-shot learning for human-AI interaction in creative tasks, emphasizing UI design to improve user experience.

Contribution

It identifies key opportunities and challenges in prompt-based learning for creative applications and proposes design goals and UI sketches to enhance user interaction.

Findings

Four design goals for prompting interfaces are proposed.

Concrete UI sketches for creative writing are illustrated.

Guidelines for developing user-friendly prompting systems are provided.

Abstract

Deep generative models have the potential to fundamentally change the way we create high-fidelity digital content but are often hard to control. Prompting a generative model is a promising recent development that in principle enables end-users to creatively leverage zero-shot and few-shot learning to assign new tasks to an AI ad-hoc, simply by writing them down. However, for the majority of end-users writing effective prompts is currently largely a trial and error process. To address this, we discuss the key opportunities and challenges for interactive creative applications that use prompting as a new paradigm for Human-AI interaction. Based on our analysis, we propose four design goals for user interfaces that support prompting. We illustrate these with concrete UI design sketches, focusing on the use case of creative writing. The research community in HCI and AI can take these as starting points to develop adequate user interfaces for models capable of zero- and few-shot learning.