TopicNet: Semantic Graph-Guided Topic Discovery

TL;DR

TopicNet is a deep hierarchical topic model that incorporates prior semantic knowledge from knowledge graphs to improve the quality and interpretability of discovered topics.

Contribution

It introduces a novel way to inject prior structural knowledge into deep hierarchical topic models using Gaussian embeddings and similarity measures.

Findings

Outperforms existing models in discovering deeper, more interpretable topics

Produces better document representations on benchmark datasets

Effectively incorporates prior semantic hierarchies into topic learning

Abstract

Existing deep hierarchical topic models are able to extract semantically meaningful topics from a text corpus in an unsupervised manner and automatically organize them into a topic hierarchy. However, it is unclear how to incorporate prior beliefs such as knowledge graph to guide the learning of the topic hierarchy. To address this issue, we introduce TopicNet as a deep hierarchical topic model that can inject prior structural knowledge as an inductive bias to influence learning. TopicNet represents each topic as a Gaussian-distributed embedding vector, projects the topics of all layers into a shared embedding space, and explores both the symmetric and asymmetric similarities between Gaussian embedding vectors to incorporate prior semantic hierarchies. With an auto-encoding variational inference network, the model parameters are optimized by minimizing the evidence lower bound and a regularization term via stochastic gradient descent. Experiments on widely used benchmarks show that TopicNet outperforms related deep topic models on discovering deeper interpretable topics and mining better document~representations.