vONTSS: vMF based semi-supervised neural topic modeling with optimal transport

TL;DR

vONTSS introduces a semi-supervised neural topic modeling approach using vMF variational autoencoders and optimal transport, effectively incorporating human knowledge to improve topic quality and classification accuracy.

Contribution

This work presents the first semi-supervised neural topic model leveraging vMF autoencoders and optimal transport, enhancing topic coherence and classification performance.

Findings

Outperforms existing semi-supervised methods in accuracy and diversity.

Supports unsupervised topic modeling with superior coherence.

Faster than state-of-the-art weakly supervised classifiers while maintaining accuracy.

Abstract

Recently, Neural Topic Models (NTM), inspired by variational autoencoders, have attracted a lot of research interest; however, these methods have limited applications in the real world due to the challenge of incorporating human knowledge. This work presents a semi-supervised neural topic modeling method, vONTSS, which uses von Mises-Fisher (vMF) based variational autoencoders and optimal transport. When a few keywords per topic are provided, vONTSS in the semi-supervised setting generates potential topics and optimizes topic-keyword quality and topic classification. Experiments show that vONTSS outperforms existing semi-supervised topic modeling methods in classification accuracy and diversity. vONTSS also supports unsupervised topic modeling. Quantitative and qualitative experiments show that vONTSS in the unsupervised setting outperforms recent NTMs on multiple aspects: vONTSS discovers highly clustered and coherent topics on benchmark datasets. It is also much faster than the state-of-the-art weakly supervised text classification method while achieving similar classification performance. We further prove the equivalence of optimal transport loss and cross-entropy loss at the global minimum.