Co-word Maps and Topic Modeling: A Comparison Using Small and Medium-Sized Corpora (n < 1000)

TL;DR

This study compares co-word mapping and topic modeling for semantic analysis in small to medium-sized corpora, finding that they produce significantly different results and are not interchangeable for such datasets.

Contribution

The paper provides an empirical comparison showing that topic modeling does not replace co-word mapping in small and medium-sized document collections.

Findings

Co-word maps and topic models are significantly uncorrelated.

Topic models reveal linguistic rather than semantic similarities.

Co-word mapping allows easier semantic interpretation.

Abstract

Induced by "big data," "topic modeling" has become an attractive alternative to mapping co-words in terms of co-occurrences and co-absences using network techniques. Does topic modeling provide an alternative for co-word mapping in research practices using moderately sized document collections? We return to the word/document matrix using first a single text with a strong argument ("The Leiden Manifesto") and then upscale to a sample of moderate size (n = 687) to study the pros and cons of the two approaches in terms of the resulting possibilities for making semantic maps that can serve an argument. The results from co-word mapping (using two different routines) versus topic modeling are significantly uncorrelated. Whereas components in the co-word maps can easily be designated, the topic models provide sets of words that are very differently organized. In these samples, the topic models seem to reveal similarities other than semantic ones (e.g., linguistic ones). In other words, topic modeling does not replace co-word mapping in small and medium-sized sets; but the paper leaves open the possibility that topic modeling would work well for the semantic mapping of large sets.