Retell, Reward, Repeat: Reinforcement Learning for Narrative Theory-Informed Story Generation

TL;DR

This paper explores reinforcement learning as a post-training method for automatic story generation, guided by narrative principles, to produce more diverse and human-aligned stories compared to traditional supervised fine-tuning.

Contribution

It introduces a reinforcement learning approach based on narrative principles, demonstrating improved diversity and alignment in story generation over supervised methods.

Findings

d-RLAIF produces more diverse stories

Stories are better aligned with human narrative conventions

Reinforcement learning is a viable alternative to supervised fine-tuning

Abstract

Despite the subjective nature of storytelling, past works on automatic story generation (ASG) have relied on limited ground truths for training and evaluation. In this work, we explore reinforcement learning (d-RLAIF) as a post-training alternative to supervised fine-tuning (SFT). We first apply Todorov's Theory of Narrative Equilibrium to establish principles that define desirable ASG qualities. We prompt 7B and 14B LLM-as-judge models with our principles to test alignment with human annotators and provide reward signals during d-RLAIF. We use Gemini-3-Flash to evaluate the output of our post-trained models and compare them to human-written stories from the TimeTravel dataset. We show that d-RLAIF offers a viable alternative to supervised fine-tuning (SFT)--producing stories that are more diverse and aligned with human narrative conventions. Our paper demonstrates the promise of reinforcement learning for linguistically grounded post-training for subjective tasks such as ASG.