An Enhanced Model-based Approach for Short Text Clustering

TL;DR

This paper introduces GSDMM+ and an improved clustering approach for short texts, effectively handling data sparsity and high dimensionality, and achieving more accurate, fine-grained clustering results with reduced noise.

Contribution

It proposes an enhanced GSDMM+ model with adaptive weighting and cluster merging, improving short text clustering accuracy and efficiency over existing methods.

Findings

GSDMM+ outperforms classical and state-of-the-art methods in clustering quality.

The approach effectively handles sparsity and high dimensionality of short texts.

Experimental results confirm the efficiency and effectiveness of the proposed methods.

Abstract

Short text clustering has become increasingly important with the popularity of social media like Twitter, Google+, and Facebook. Existing methods can be broadly categorized into two paradigms: topic model-based approaches and deep representation learning-based approaches. This task is inherently challenging due to the sparse, large-scale, and high-dimensional characteristics of the short text data. Furthermore, the computational intensity required by representation learning significantly increases the running time. To address these issues, we propose a collapsed Gibbs Sampling algorithm for the Dirichlet Multinomial Mixture model (GSDMM), which effectively handles the sparsity and high dimensionality of short texts while identifying representative words for each cluster. Based on several aspects of GSDMM that warrant further refinement, we propose an improved approach, GSDMM+, designed to further optimize its performance. GSDMM+ reduces initialization noise and adaptively adjusts word weights based on entropy, achieving fine-grained clustering that reveals more topic-related information. Additionally, strategic cluster merging is employed to refine clustering granularity, better aligning the predicted distribution with the true category distribution. We conduct extensive experiments, comparing our methods with both classical and state-of-the-art approaches. The experimental results demonstrate the efficiency and effectiveness of our methods. The source code for our model is publicly available at https://github.com/chehaoa/VEMC.