The GPT-WritingPrompts Dataset: A Comparative Analysis of Character Portrayal in Short Stories

TL;DR

This paper introduces an augmented dataset of GPT-3.5 generated stories based on Reddit prompts, analyzing differences in emotional and descriptive features compared to human stories across six dimensions.

Contribution

It provides a comparative analysis of human and GPT-generated stories, highlighting key differences and biases in storytelling characteristics.

Findings

Generated stories differ significantly from human stories across all six analyzed dimensions.

Humans and machines show similar biases related to narrative point-of-view and protagonist gender.

The dataset and analysis tools are publicly released for further research.

Abstract

The improved generative capabilities of large language models have made them a powerful tool for creative writing and storytelling. It is therefore important to quantitatively understand the nature of generated stories, and how they differ from human storytelling. We augment the Reddit WritingPrompts dataset with short stories generated by GPT-3.5, given the same prompts. We quantify and compare the emotional and descriptive features of storytelling from both generative processes, human and machine, along a set of six dimensions. We find that generated stories differ significantly from human stories along all six dimensions, and that human and machine generations display similar biases when grouped according to the narrative point-of-view and gender of the main protagonist. We release our dataset and code at https://github.com/KristinHuangg/gpt-writing-prompts.