Cluster Based Symbolic Representation for Skewed Text Categorization

TL;DR

This paper introduces a clustering-based symbolic representation method to balance skewed text datasets, improving classification efficiency and accuracy on benchmark datasets compared to existing models.

Contribution

It presents a novel approach combining clustering and symbolic representation to address class imbalance in text categorization, reducing dimensionality and computational cost.

Findings

Outperforms existing models on Reuters 21578 and TDT2 datasets.

Reduces classification time and space requirements.

Effective in handling imbalanced text corpora.

Abstract

In this work, a problem associated with imbalanced text corpora is addressed. A method of converting an imbalanced text corpus into a balanced one is presented. The presented method employs a clustering algorithm for conversion. Initially to avoid curse of dimensionality, an effective representation scheme based on term class relevancy measure is adapted, which drastically reduces the dimension to the number of classes in the corpus. Subsequently, the samples of larger sized classes are grouped into a number of subclasses of smaller sizes to make the entire corpus balanced. Each subclass is then given a single symbolic vector representation by the use of interval valued features. This symbolic representation in addition to being compact helps in reducing the space requirement and also the classification time. The proposed model has been empirically demonstrated for its superiority on bench marking datasets viz., Reuters 21578 and TDT2. Further, it has been compared against several other existing contemporary models including model based on support vector machine. The comparative analysis indicates that the proposed model outperforms the other existing models.