Neural Topic Modeling by Incorporating Document Relationship Graph

TL;DR

This paper introduces Graph Topic Model (GTM), a neural topic modeling approach utilizing graph neural networks to incorporate document relationships, improving topical representations by leveraging shared words and document connections.

Contribution

The paper presents a novel GNN-based neural topic model that integrates document relationship graphs to enhance topic discovery in corpora.

Findings

GTM outperforms baseline models on three datasets.

Graph structure improves topical coherence.

Document relationships enrich topic representations.

Abstract

Graph Neural Networks (GNNs) that capture the relationships between graph nodes via message passing have been a hot research direction in the natural language processing community. In this paper, we propose Graph Topic Model (GTM), a GNN based neural topic model that represents a corpus as a document relationship graph. Documents and words in the corpus become nodes in the graph and are connected based on document-word co-occurrences. By introducing the graph structure, the relationships between documents are established through their shared words and thus the topical representation of a document is enriched by aggregating information from its neighboring nodes using graph convolution. Extensive experiments on three datasets were conducted and the results demonstrate the effectiveness of the proposed approach.