Concentrated Document Topic Model

TL;DR

The paper introduces the Concentrated Document Topic Model (CDTM), which uses an entropy penalty to produce sparse, coherent, and concentrated topic distributions in unsupervised text classification, outperforming LDA on benchmark data.

Contribution

The novel CDTM model applies an exponential entropy penalty to enhance topic concentration and sparsity in document-topic distributions, improving over existing models like LDA.

Findings

More coherent topics than LDA

Produces more concentrated and sparse distributions

Effective on benchmark NIPS dataset

Abstract

We propose a Concentrated Document Topic Model(CDTM) for unsupervised text classification, which is able to produce a concentrated and sparse document topic distribution. In particular, an exponential entropy penalty is imposed on the document topic distribution. Documents that have diverse topic distributions are penalized more, while those having concentrated topics are penalized less. We apply the model to the benchmark NIPS dataset and observe more coherent topics and more concentrated and sparse document-topic distributions than Latent Dirichlet Allocation(LDA).