Learning Topics using Semantic Locality

TL;DR

This paper introduces a novel feature extraction method for topic modeling that enhances the semantic quality of topics by filtering and merging word pairs, leading to improved accuracy over existing models.

Contribution

A new three-step feature extraction technique for topic modeling that improves topic quality by semantic filtering and merging of word pairs.

Findings

Improves topic accuracy by up to 12.99%.

Effective on datasets like OMDb, Reuters, and 20NewsGroup.

Outperforms LDA and RBM in experiments.

Abstract

The topic modeling discovers the latent topic probability of the given text documents. To generate the more meaningful topic that better represents the given document, we proposed a new feature extraction technique which can be used in the data preprocessing stage. The method consists of three steps. First, it generates the word/word-pair from every single document. Second, it applies a two-way TF-IDF algorithm to word/word-pair for semantic filtering. Third, it uses the K-means algorithm to merge the word pairs that have the similar semantic meaning. Experiments are carried out on the Open Movie Database (OMDb), Reuters Dataset and 20NewsGroup Dataset. The mean Average Precision score is used as the evaluation metric. Comparing our results with other state-of-the-art topic models, such as Latent Dirichlet allocation and traditional Restricted Boltzmann Machines. Our proposed data preprocessing can improve the generated topic accuracy by up to 12.99\%.