PromptHive: Bringing Subject Matter Experts Back to the Forefront with Collaborative Prompt Engineering for Educational Content Creation

TL;DR

PromptHive is a collaborative interface that empowers subject matter experts to efficiently craft and refine prompts for educational content, significantly reducing development time and enhancing output quality.

Contribution

This work introduces PromptHive, a novel tool that facilitates systematic prompt iteration and collaboration among non-AI experts for educational content creation.

Findings

Reduced authoring time from months to hours

Lowered perceived cognitive load by half

Generated content comparable to human-authored materials

Abstract

Involving subject matter experts in prompt engineering can guide LLM outputs toward more helpful, accurate, and tailored content that meets the diverse needs of different domains. However, iterating towards effective prompts can be challenging without adequate interface support for systematic experimentation within specific task contexts. In this work, we introduce PromptHive, a collaborative interface for prompt authoring, designed to better connect domain knowledge with prompt engineering through features that encourage rapid iteration on prompt variations. We conducted an evaluation study with ten subject matter experts in math and validated our design through two collaborative prompt-writing sessions and a learning gain study with 358 learners. Our results elucidate the prompt iteration process and validate the tool's usability, enabling non-AI experts to craft prompts that generate content comparable to human-authored materials while reducing perceived cognitive load by half and shortening the authoring process from several months to just a few hours.