Prompting a Large Language Model to Generate Diverse Motivational Messages: A Comparison with Human-Written Messages

TL;DR

This paper compares the diversity of motivational messages generated by GPT-4 using crowdsourcing-inspired prompts versus baseline prompts, finding that the former yields more diverse outputs and discussing implications for human and AI-generated content.

Contribution

It demonstrates that prompts modeled after crowdsourcing tasks can significantly enhance the diversity of messages generated by large language models.

Findings

GPT-4 with crowdsourcing prompts produces more diverse messages.

Baseline prompts result in less diverse outputs.

Implications for using LLMs in creative content generation.

Abstract

Large language models (LLMs) are increasingly capable and prevalent, and can be used to produce creative content. The quality of content is influenced by the prompt used, with more specific prompts that incorporate examples generally producing better results. On from this, it could be seen that using instructions written for crowdsourcing tasks (that are specific and include examples to guide workers) could prove effective LLM prompts. To explore this, we used a previous crowdsourcing pipeline that gave examples to people to help them generate a collectively diverse corpus of motivational messages. We then used this same pipeline to generate messages using GPT-4, and compared the collective diversity of messages from: (1) crowd-writers, (2) GPT-4 using the pipeline, and (3 & 4) two baseline GPT-4 prompts. We found that the LLM prompts using the crowdsourcing pipeline caused GPT-4 to produce more diverse messages than the two baseline prompts. We also discuss implications from messages generated by both human writers and LLMs.