Clustering Words by Projection Entropy

TL;DR

This paper demonstrates how entropy agglomeration (EA), a new clustering algorithm based on projection entropy, effectively groups words in a literary text by analyzing their occurrences across paragraphs.

Contribution

It introduces the application of entropy agglomeration (EA) to text clustering, showcasing its ability to capture meaningful word relationships using a novel entropy measure.

Findings

EA successfully clusters related words in the text.

The method captures significant semantic relationships.

Implementation is available as open-source software.

Abstract

We apply entropy agglomeration (EA), a recently introduced algorithm, to cluster the words of a literary text. EA is a greedy agglomerative procedure that minimizes projection entropy (PE), a function that can quantify the segmentedness of an element set. To apply it, the text is reduced to a feature allocation, a combinatorial object to represent the word occurences in the text's paragraphs. The experiment results demonstrate that EA, despite its reduction and simplicity, is useful in capturing significant relationships among the words in the text. This procedure was implemented in Python and published as a free software: REBUS.