Towards a Linguistic Evaluation of Narratives: A Quantitative Stylistic Framework

TL;DR

This paper introduces a quantitative linguistic framework for evaluating narrative quality, using linguistic features to distinguish professional from self-published texts and outperform traditional metrics.

Contribution

It presents a novel methodology for automatic narrative assessment based on comprehensive linguistic features, validated on a diverse corpus.

Findings

Successfully clustered narratives into professional and self-published groups.

Outperformed traditional story evaluation metrics.

Validated against human annotations with significant accuracy.

Abstract

The evaluation of narrative quality remains a complex challenge, as it involves subjective factors such as plot, character development, and emotional impact. This work proposes a quantitative approach to narrative assessment by focusing on the linguistic dimension as a primary indicator of quality. The paper presents a methodology for the automatic evaluation of narrative based on the extraction of a comprehensive set of 33 quantitative linguistic features categorized into lexical, syntactic, and semantic groups. To test the model, an experiment was conducted on a specialized corpus of 23 books, including canonical masterpieces and self-published works. Through a similarity matrix, the system successfully clustered the narratives, distinguishing almost perfectly between professionally edited and self-published texts. Furthermore, the methodology was validated against a human-annotated dataset; it significantly outperforms traditional story-level evaluation metrics, demonstrating the effectiveness of quantitative linguistic features in assessing narrative quality.