TopicTag: Automatic Annotation of NMF Topic Models Using Chain of Thought and Prompt Tuning with LLMs

TL;DR

This paper proposes an automated method for labeling topics generated by NMF in large text datasets by leveraging large language models and prompt engineering, reducing manual effort and improving organization.

Contribution

It introduces a novel approach combining NMF with LLMs and prompt engineering for automatic topic labeling, enhancing the efficiency of topic modeling workflows.

Findings

Effective labeling of 34,000 scientific abstracts

Improved organization and knowledge management

Reduction in manual labeling effort

Abstract

Topic modeling is a technique for organizing and extracting themes from large collections of unstructured text. Non-negative matrix factorization (NMF) is a common unsupervised approach that decomposes a term frequency-inverse document frequency (TF-IDF) matrix to uncover latent topics and segment the dataset accordingly. While useful for highlighting patterns and clustering documents, NMF does not provide explicit topic labels, necessitating subject matter experts (SMEs) to assign labels manually. We present a methodology for automating topic labeling in documents clustered via NMF with automatic model determination (NMFk). By leveraging the output of NMFk and employing prompt engineering, we utilize large language models (LLMs) to generate accurate topic labels. Our case study on over 34,000 scientific abstracts on Knowledge Graphs demonstrates the effectiveness of our method in enhancing knowledge management and document organization.