A Novel Document Generation Process for Topic Detection based on Hierarchical Latent Tree Models

TL;DR

This paper introduces a hierarchical latent tree model-based document generation process that improves topic detection by better capturing word counts and producing more meaningful topic hierarchies, outperforming LDA-based methods.

Contribution

The paper presents a novel hierarchical latent tree model for document generation, achieving superior fit and more meaningful topics compared to existing LDA-based approaches.

Findings

Achieves better model fit with fewer parameters.

Produces more meaningful topics and hierarchies.

Sets new state-of-the-art in hierarchical topic detection.

Abstract

We propose a novel document generation process based on hierarchical latent tree models (HLTMs) learned from data. An HLTM has a layer of observed word variables at the bottom and multiple layers of latent variables on top. For each document, we first sample values for the latent variables layer by layer via logic sampling, then draw relative frequencies for the words conditioned on the values of the latent variables, and finally generate words for the document using the relative word frequencies. The motivation for the work is to take word counts into consideration with HLTMs. In comparison with LDA-based hierarchical document generation processes, the new process achieves drastically better model fit with much fewer parameters. It also yields more meaningful topics and topic hierarchies. It is the new state-of-the-art for the hierarchical topic detection.