Memory and Knowledge Augmented Language Models for Inferring Salience in Long-Form Stories

TL;DR

This paper enhances language models with external knowledge and memory mechanisms to better detect salient events in long-form stories, using unsupervised methods and literary summaries for evaluation.

Contribution

It introduces a novel approach combining knowledge bases and memory modules into transformer models for improved story salience detection.

Findings

Knowledge and memory integration improves salience detection accuracy.

The model outperforms baseline transformer models without these enhancements.

Evaluation on literary summaries demonstrates significant performance gains.

Abstract

Measuring event salience is essential in the understanding of stories. This paper takes a recent unsupervised method for salience detection derived from Barthes Cardinal Functions and theories of surprise and applies it to longer narrative forms. We improve the standard transformer language model by incorporating an external knowledgebase (derived from Retrieval Augmented Generation) and adding a memory mechanism to enhance performance on longer works. We use a novel approach to derive salience annotation using chapter-aligned summaries from the Shmoop corpus for classic literary works. Our evaluation against this data demonstrates that our salience detection model improves performance over and above a non-knowledgebase and memory augmented language model, both of which are crucial to this improvement.