Narrative Text Generation with a Latent Discrete Plan

TL;DR

This paper introduces a deep latent variable model for story generation that learns to produce coherent stories by inducing latent anchor word sequences, improving story quality and diversity without relying on supervised plan annotations.

Contribution

The paper presents an unsupervised latent plan induction model for story generation, utilizing variational inference and anchor words to guide story coherence without external plan supervision.

Findings

Human evaluations favor the proposed model over baselines without plans.

Model achieves better perplexity and diversity scores.

Effective control of story content via learned discrete plans.

Abstract

Past work on story generation has demonstrated the usefulness of conditioning on a generation plan to generate coherent stories. However, these approaches have used heuristics or off-the-shelf models to first tag training stories with the desired type of plan, and then train generation models in a supervised fashion. In this paper, we propose a deep latent variable model that first samples a sequence of anchor words, one per sentence in the story, as part of its generative process. During training, our model treats the sequence of anchor words as a latent variable and attempts to induce anchoring sequences that help guide generation in an unsupervised fashion. We conduct experiments with several types of sentence decoder distributions: left-to-right and non-monotonic, with different degrees of restriction. Further, since we use amortized variational inference to train our model, we introduce two corresponding types of inference network for predicting the posterior on anchor words. We conduct human evaluations which demonstrate that the stories produced by our model are rated better in comparison with baselines which do not consider story plans, and are similar or better in quality relative to baselines which use external supervision for plans. Additionally, the proposed model gets favorable scores when evaluated on perplexity, diversity, and control of story via discrete plan.