A Novel Method of Fuzzy Topic Modeling based on Transformer Processing

TL;DR

This paper introduces a fuzzy topic modeling approach utilizing transformer-based document embeddings and soft clustering, providing more intuitive and natural topic results compared to traditional LDA in market trend analysis.

Contribution

It proposes a novel fuzzy topic modeling method that combines transformer embeddings with soft clustering, addressing limitations of LDA in interpretability and manual topic number selection.

Findings

Fuzzy topic modeling yields more natural topics than LDA.

Transformer embeddings improve topic coherence.

Method applied successfully in press release monitoring.

Abstract

Topic modeling is admittedly a convenient way to monitor markets trend. Conventionally, Latent Dirichlet Allocation, LDA, is considered a must-do model to gain this type of information. By given the merit of deducing keyword with token conditional probability in LDA, we can know the most possible or essential topic. However, the results are not intuitive because the given topics cannot wholly fit human knowledge. LDA offers the first possible relevant keywords, which also brings out another problem of whether the connection is reliable based on the statistic possibility. It is also hard to decide the topic number manually in advance. As the booming trend of using fuzzy membership to cluster and using transformers to embed words, this work presents the fuzzy topic modeling based on soft clustering and document embedding from state-of-the-art transformer-based model. In our practical application in a press release monitoring, the fuzzy topic modeling gives a more natural result than the traditional output from LDA.