Modeling Word Relatedness in Latent Dirichlet Allocation

TL;DR

This paper introduces WR-LDA, an extension of LDA that incorporates word correlations, enabling better handling of infrequent words and multi-language topics, with demonstrated improved performance.

Contribution

The paper proposes WR-LDA, a novel model that integrates word correlation into LDA, enhancing its capabilities beyond standard topic modeling.

Findings

WR-LDA outperforms standard LDA in experiments

Enables modeling of infrequent words

Supports multi-language topic modeling

Abstract

Standard LDA model suffers the problem that the topic assignment of each word is independent and word correlation hence is neglected. To address this problem, in this paper, we propose a model called Word Related Latent Dirichlet Allocation (WR-LDA) by incorporating word correlation into LDA topic models. This leads to new capabilities that standard LDA model does not have such as estimating infrequently occurring words or multi-language topic modeling. Experimental results demonstrate the effectiveness of our model compared with standard LDA.