Word Embeddings and Validity Indexes in Fuzzy Clustering

TL;DR

This paper investigates the effectiveness of fuzzy clustering algorithms on word embeddings derived from COVID-related text data, introducing hybrid methods and analyzing their sensitivity to high-dimensional data using validity indexes.

Contribution

It introduces hybrid fuzzy clustering methods combining evolutionary algorithms with traditional fuzzy clustering, and evaluates their performance on word embeddings.

Findings

Fuzzy clustering sensitivity to high-dimensional data is significant.

Parameter tuning greatly affects clustering performance.

Validity indexes help compare different clustering approaches.

Abstract

In the new era of internet systems and applications, a concept of detecting distinguished topics from huge amounts of text has gained a lot of attention. These methods use representation of text in a numerical format -- called embeddings -- to imitate human-based semantic similarity between words. In this study, we perform a fuzzy-based analysis of various vector representations of words, i.e., word embeddings. Also we introduce new methods of fuzzy clustering based on hybrid implementation of fuzzy clustering methods with an evolutionary algorithm named Forest Optimization. We use two popular fuzzy clustering algorithms on count-based word embeddings, with different methods and dimensionality. Words about covid from Kaggle dataset gathered and calculated into vectors and clustered. The results indicate that fuzzy clustering algorithms are very sensitive to high-dimensional data, and parameter tuning can dramatically change their performance. We evaluate results of experiments with various clustering validity indexes to compare different algorithm variation with different embeddings accuracy.