On Large-Scale Dynamic Topic Modeling with Nonnegative CP Tensor Decomposition

TL;DR

This paper introduces a novel application of nonnegative CP tensor decomposition for large-scale dynamic topic modeling, effectively capturing temporal information and outperforming traditional NMF-based methods.

Contribution

It is the first to apply NNCPD to dynamic topic modeling, offering a more effective way to analyze temporal data by directly decomposing data tensors.

Findings

NNCPD significantly outperforms NMF-based methods on synthetic data.

Application to real data demonstrates improved topic evolution tracking.

The approach preserves temporal information more effectively.

Abstract

There is currently an unprecedented demand for large-scale temporal data analysis due to the explosive growth of data. Dynamic topic modeling has been widely used in social and data sciences with the goal of learning latent topics that emerge, evolve, and fade over time. Previous work on dynamic topic modeling primarily employ the method of nonnegative matrix factorization (NMF), where slices of the data tensor are each factorized into the product of lower-dimensional nonnegative matrices. With this approach, however, information contained in the temporal dimension of the data is often neglected or underutilized. To overcome this issue, we propose instead adopting the method of nonnegative CANDECOMP/PARAPAC (CP) tensor decomposition (NNCPD), where the data tensor is directly decomposed into a minimal sum of outer products of nonnegative vectors, thereby preserving the temporal information. The viability of NNCPD is demonstrated through application to both synthetic and real data, where significantly improved results are obtained compared to those of typical NMF-based methods. The advantages of NNCPD over such approaches are studied and discussed. To the best of our knowledge, this is the first time that NNCPD has been utilized for the purpose of dynamic topic modeling, and our findings will be transformative for both applications and further developments.