Improving Neural Story Generation by Targeted Common Sense Grounding

TL;DR

This paper introduces a multi-task learning approach with auxiliary datasets to enhance common sense reasoning in neural story generation, resulting in more coherent stories and improved perplexity scores.

Contribution

It presents a novel multi-task training scheme combined with a two-stage fine-tuning pipeline to improve common sense grounding in neural language models for story generation.

Findings

Achieved state-of-the-art perplexity on the Writing Prompts dataset.

Significantly improved common sense reasoning in generated stories.

Demonstrated the effectiveness of auxiliary datasets in grounding language models.

Abstract

Stories generated with neural language models have shown promise in grammatical and stylistic consistency. However, the generated stories are still lacking in common sense reasoning, e.g., they often contain sentences deprived of world knowledge. We propose a simple multi-task learning scheme to achieve quantitatively better common sense reasoning in language models by leveraging auxiliary training signals from datasets designed to provide common sense grounding. When combined with our two-stage fine-tuning pipeline, our method achieves improved common sense reasoning and state-of-the-art perplexity on the Writing Prompts (Fan et al., 2018) story generation dataset.