TopiCLEAR: Topic extraction by CLustering Embeddings with Adaptive dimensional Reduction

TL;DR

TopiCLEAR is a novel clustering-based method that extracts interpretable topics from short social media texts by combining embedding, adaptive dimensionality reduction, and iterative refinement, outperforming existing approaches.

Contribution

It introduces a new unsupervised topic extraction technique that operates directly on raw text, avoiding preprocessing and improving interpretability and accuracy on social media data.

Findings

Achieves highest similarity to human-labeled topics among tested methods.

Significantly outperforms baseline models on diverse datasets.

Produces more interpretable and meaningful topics.

Abstract

Rapid expansion of social media platforms such as X (formerly Twitter), Facebook, and Reddit has enabled large-scale analysis of public perceptions on diverse topics, including social issues, politics, natural disasters, and consumer sentiment. Topic modeling is a widely used approach for uncovering latent themes in text data, typically framed as an unsupervised classification task. However, traditional models, originally designed for longer and more formal documents, struggle with short social media posts due to limited co-occurrence statistics, fragmented semantics, inconsistent spelling, and informal language. To address these challenges, we propose a new method, TopiCLEAR: Topic extraction by CLustering Embeddings with Adaptive dimensional Reduction. Specifically, each text is embedded using Sentence-BERT (SBERT) and provisionally clustered using Gaussian Mixture Models (GMM). The clusters are then refined iteratively using a supervised projection based on linear discriminant analysis, followed by GMM-based clustering until convergence. Notably, our method operates directly on raw text, eliminating the need for preprocessing steps such as stop word removal. We evaluate our approach on four diverse datasets, 20News, AgNewsTitle, Reddit, and TweetTopic, each containing human-labeled topic information. Compared with seven baseline methods, including a recent SBERT-based method and a zero-shot generative AI method, our approach achieves the highest similarity to human-annotated topics, with significant improvements for both social media posts and online news articles. Additionally, qualitative analysis shows that our method produces more interpretable topics, highlighting its potential for applications in social media data and web content analytics.