Comparing intermittency and network measurements of words and their dependency on authorship

TL;DR

This study investigates how topological properties of word co-occurrence networks and intermittency in word distribution vary with author style, using statistical and network analysis on 40 books, and explores their use in authorship recognition.

Contribution

It quantifies the dependency of network and intermittency features on authorship and evaluates their effectiveness in authorship attribution using machine learning.

Findings

Skewness in word intermittency distribution strongly depends on authorship.

Average shortest path length in networks varies significantly with author.

Combining network and intermittency features yields about 65% accuracy in authorship recognition.

Abstract

Many features from texts and languages can now be inferred from statistical analyses using concepts from complex networks and dynamical systems. In this paper we quantify how topological properties of word co-occurrence networks and intermittency (or burstiness) in word distribution depend on the style of authors. Our database contains 40 books from 8 authors who lived in the 19th and 20th centuries, for which the following network measurements were obtained: clustering coefficient, average shortest path lengths, and betweenness. We found that the two factors with stronger dependency on the authors were the skewness in the distribution of word intermittency and the average shortest paths. Other factors such as the betweeness and the Zipf's law exponent show only weak dependency on authorship. Also assessed was the contribution from each measurement to authorship recognition using three machine learning methods. The best performance was a ca. 65 % accuracy upon combining complex network and intermittency features with the nearest neighbor algorithm. From a detailed analysis of the interdependence of the various metrics it is concluded that the methods used here are complementary for providing short- and long-scale perspectives of texts, which are useful for applications such as identification of topical words and information retrieval.