TopNet: Learning from Neural Topic Model to Generate Long Stories

TL;DR

TopNet leverages neural topic modeling to generate story skeletons from short inputs, significantly improving long story generation quality by addressing information sparsity.

Contribution

It introduces a novel framework that maps short texts to topic distributions and uses them to generate story skeletons, enhancing long story generation.

Findings

Outperforms state-of-the-art models in automatic evaluation.

Achieves higher human evaluation scores.

Effectively selects skeleton words for long stories.

Abstract

Long story generation (LSG) is one of the coveted goals in natural language processing. Different from most text generation tasks, LSG requires to output a long story of rich content based on a much shorter text input, and often suffers from information sparsity. In this paper, we propose \emph{TopNet} to alleviate this problem, by leveraging the recent advances in neural topic modeling to obtain high-quality skeleton words to complement the short input. In particular, instead of directly generating a story, we first learn to map the short text input to a low-dimensional topic distribution (which is pre-assigned by a topic model). Based on this latent topic distribution, we can use the reconstruction decoder of the topic model to sample a sequence of inter-related words as a skeleton for the story. Experiments on two benchmark datasets show that our proposed framework is highly effective in skeleton word selection and significantly outperforms the state-of-the-art models in both automatic evaluation and human evaluation.