How to predict creativity ratings from written narratives: A comparison of co-occurrence and textual forma mentis networks

TL;DR

This paper compares co-occurrence and textual forma mentis networks for predicting creativity ratings from short texts, demonstrating TFMNs' superior performance and providing a practical, reproducible workflow for researchers.

Contribution

It introduces and evaluates a workflow for using semantic networks, especially TFMNs, to predict creativity ratings, highlighting their advantages over co-occurrence networks.

Findings

TFMNs outperform co-occurrence networks in prediction accuracy.

Network structural features are most predictive of creativity ratings.

Emotion features are less effective than structural network features.

Abstract

This tutorial paper provides a step-by-step workflow for building and analysing semantic networks from short creative texts. We introduce and compare two widely used text-to-network approaches: word co-occurrence networks and textual forma mentis networks (TFMNs). We also demonstrate how they can be used in machine learning to predict human creativity ratings. Using a corpus of 1029 short stories, we guide readers through text preprocessing, network construction, feature extraction (structural measures, spreading-activation indices, and emotion scores), and application of regression models. We evaluate how network-construction choices influence both network topology and predictive performance. Across all modelling settings, TFMNs consistently outperformed co-occurrence networks through lower prediction errors (best MAE = 0.581 for TFMN, vs 0.592 for co-occurrence with window size 3). Network-structural features dominated predictive performance (MAE = 0.591 for TFMN), whereas emotion features performed worse (MAE = 0.711 for TFMN) and spreading-activation measures contributed little (MAE = 0.788 for TFMN). This paper offers practical guidance for researchers interested in applying network-based methods for cognitive fields like creativity research. we show when syntactic networks are preferable to surface co-occurrence models, and provide an open, reproducible workflow accessible to newcomers in the field, while also offering deeper methodological insight for experienced researchers.