Controllable Neural Story Plot Generation via Reward Shaping

TL;DR

This paper introduces a reward-shaping method that guides pre-trained language models to generate coherent and goal-oriented story plots, improving both automated and human-evaluated story quality.

Contribution

It proposes a novel reward-shaping technique that incorporates intermediate rewards from story corpora into language models to control story plot generation.

Findings

Generated plots better achieve specified goals.

Stories have more plausible event sequences.

Method outperforms baseline techniques in coherence.

Abstract

Language-modeling--based approaches to story plot generation attempt to construct a plot by sampling from a language model (LM) to predict the next character, word, or sentence to add to the story. LM techniques lack the ability to receive guidance from the user to achieve a specific goal, resulting in stories that don't have a clear sense of progression and lack coherence. We present a reward-shaping technique that analyzes a story corpus and produces intermediate rewards that are backpropagated into a pre-trained LM in order to guide the model towards a given goal. Automated evaluations show our technique can create a model that generates story plots which consistently achieve a specified goal. Human-subject studies show that the generated stories have more plausible event ordering than baseline plot generation techniques.