Using LLM-Based Approaches to Enhance and Automate Topic Labeling

TL;DR

This paper investigates using Large Language Models to automate and improve topic labeling after topic modeling, introducing a new metric for evaluating label quality based on semantic representativeness.

Contribution

It presents a novel approach combining BERTopic with LLMs for label generation and proposes a new quantitative metric for label evaluation.

Findings

LLMs can generate more meaningful topic labels than manual methods.

Different keyword and summary selection strategies impact label quality.

The proposed metric effectively measures semantic representativeness of labels.

Abstract

Topic modeling has become a crucial method for analyzing text data, particularly for extracting meaningful insights from large collections of documents. However, the output of these models typically consists of lists of keywords that require manual interpretation for precise labeling. This study explores the use of Large Language Models (LLMs) to automate and enhance topic labeling by generating more meaningful and contextually appropriate labels. After applying BERTopic for topic modeling, we explore different approaches to select keywords and document summaries within each topic, which are then fed into an LLM to generate labels. Each approach prioritizes different aspects, such as dominant themes or diversity, to assess their impact on label quality. Additionally, recognizing the lack of quantitative methods for evaluating topic labels, we propose a novel metric that measures how semantically representative a label is of all documents within a topic.