Clustering Unstructured Data (Flat Files) - An Implementation in Text Mining Tool

TL;DR

This paper presents an implementation of the k-Means clustering algorithm tailored for unstructured text documents, including a novel document representation technique to enhance clustering quality.

Contribution

It introduces a specific implementation of k-Means for text data and proposes a new document representation method to improve clustering results.

Findings

Effective clustering of unstructured text documents.

Proposed document representation improves cluster quality.

Implementation details facilitate practical application.

Abstract

With the advancement of technology and reduced storage costs, individuals and organizations are tending towards the usage of electronic media for storing textual information and documents. It is time consuming for readers to retrieve relevant information from unstructured document collection. It is easier and less time consuming to find documents from a large collection when the collection is ordered or classified by group or category. The problem of finding best such grouping is still there. This paper discusses the implementation of k-Means clustering algorithm for clustering unstructured text documents that we implemented, beginning with the representation of unstructured text and reaching the resulting set of clusters. Based on the analysis of resulting clusters for a sample set of documents, we have also proposed a technique to represent documents that can further improve the clustering result.