Skeleton-based Coherence Modeling in Narratives

TL;DR

This paper investigates the use of skeleton consistency across sentences as a metric for textual coherence, proposing a new neural network model that outperforms traditional similarity measures.

Contribution

It introduces the Sentence/Skeleton Similarity Network (SSN) for coherence modeling and compares its effectiveness against baseline similarity techniques.

Findings

SSN outperforms cosine similarity and Euclidean distance in coherence tasks.

Sentence-level models outperform skeleton-based models in coherence evaluation.

Skeletons show promise but current sentence-level approaches are more effective.

Abstract

Modeling coherence in text has been a task that has excited NLP researchers since a long time. It has applications in detecting incoherent structures and helping the author fix them. There has been recent work in using neural networks to extract a skeleton from one sentence, and then use that skeleton to generate the next sentence for coherent narrative story generation. In this project, we aim to study if the consistency of skeletons across subsequent sentences is a good metric to characterize the coherence of a given body of text. We propose a new Sentence/Skeleton Similarity Network (SSN) for modeling coherence across pairs of sentences, and show that this network performs much better than baseline similarity techniques like cosine similarity and Euclidean distance. Although skeletons appear to be promising candidates for modeling coherence, our results show that sentence-level models outperform those on skeletons for evaluating textual coherence, thus indicating that the current state-of-the-art coherence modeling techniques are going in the right direction by dealing with sentences rather than their sub-parts.