Experimental Estimation of Number of Clusters Based on Cluster Quality

TL;DR

This paper investigates an experimental method to estimate the optimal number of clusters in text clustering by evaluating cluster quality, addressing the common requirement of predefining cluster count in algorithms.

Contribution

It introduces an experimental approach to determine the number of clusters based on cluster quality metrics, specifically for partitional clustering algorithms in text mining.

Findings

Cluster quality metrics can effectively estimate the optimal number of clusters.

The approach improves clustering organization without prior knowledge of cluster count.

Experimental results validate the method's applicability to large text datasets.

Abstract

Text Clustering is a text mining technique which divides the given set of text documents into significant clusters. It is used for organizing a huge number of text documents into a well-organized form. In the majority of the clustering algorithms, the number of clusters must be specified apriori, which is a drawback of these algorithms. The aim of this paper is to show experimentally how to determine the number of clusters based on cluster quality. Since partitional clustering algorithms are well-suited for clustering large document datasets, we have confined our analysis to a partitional clustering algorithm.