Enhancing Topic Interpretability for Neural Topic Modeling through Topic-wise Contrastive Learning

TL;DR

This paper introduces ContraTopic, a neural topic model that uses contrastive learning to improve the interpretability of discovered topics, addressing the gap between likelihood maximization and interpretability in traditional NTMs.

Contribution

The paper proposes a novel NTM framework with a differentiable regularizer based on topic-wise contrastive learning to enhance topic interpretability.

Findings

ContraTopic outperforms state-of-the-art NTMs in interpretability across three datasets.

The contrastive regularizer improves internal coherence and external distinction of topics.

Experiments show consistent interpretability improvements without sacrificing data likelihood.

Abstract

Data mining and knowledge discovery are essential aspects of extracting valuable insights from vast datasets. Neural topic models (NTMs) have emerged as a valuable unsupervised tool in this field. However, the predominant objective in NTMs, which aims to discover topics maximizing data likelihood, often lacks alignment with the central goals of data mining and knowledge discovery which is to reveal interpretable insights from large data repositories. Overemphasizing likelihood maximization without incorporating topic regularization can lead to an overly expansive latent space for topic modeling. In this paper, we present an innovative approach to NTMs that addresses this misalignment by introducing contrastive learning measures to assess topic interpretability. We propose a novel NTM framework, named ContraTopic, that integrates a differentiable regularizer capable of evaluating multiple facets of topic interpretability throughout the training process. Our regularizer adopts a unique topic-wise contrastive methodology, fostering both internal coherence within topics and clear external distinctions among them. Comprehensive experiments conducted on three diverse datasets demonstrate that our approach consistently produces topics with superior interpretability compared to state-of-the-art NTMs.