PRISM: LLM-Guided Semantic Clustering for High-Precision Topics

TL;DR

PRISM is a novel framework that combines large language models with semantic clustering to improve topic discovery and separation in text corpora efficiently and interpretably.

Contribution

It introduces a student-teacher pipeline for distilling LLM supervision into lightweight models and demonstrates effective web-scale text analysis.

Findings

PRISM outperforms state-of-the-art local topic models in topic separability.

Requires only a small number of LLM queries for training.

Enables interpretable, locally deployable web-scale text analysis.

Abstract

In this paper, we propose Precision-Informed Semantic Modeling (PRISM), a structured topic modeling framework combining the benefits of rich representations captured by LLMs with the low cost and interpretability of latent semantic clustering methods. PRISM fine-tunes a sentence encoding model using a sparse set of LLM- provided labels on samples drawn from some corpus of interest. We segment this embedding space with thresholded clustering, yielding clusters that separate closely related topics within some narrow domain. Across multiple corpora, PRISM improves topic separability over state-of-the-art local topic models and even over clustering on large, frontier embedding models while requiring only a small number of LLM queries to train. This work contributes to several research streams by providing (i) a student-teacher pipeline to distill sparse LLM supervision into a lightweight model for topic discovery; (ii) an analysis of the efficacy of sampling strategies to improve local geometry for cluster separability; and (iii) an effective approach for web-scale text analysis, enabling researchers and practitioners to track nuanced claims and subtopics online with an interpretable, locally deployable framework.