Discovering Multi-Scale Semantic Structure in Text Corpora Using Density-Based Trees and LLM Embeddings

TL;DR

This paper introduces a novel hierarchical density modeling approach using LLM embeddings to uncover multi-scale semantic structures in large text corpora, enabling explicit, interpretable topic relationships without fixed taxonomies.

Contribution

It operationalizes hierarchical density modeling on LLM embeddings, revealing multi-scale semantic structures through a progressive relaxation of density constraints, unlike traditional fixed taxonomy methods.

Findings

Semantic alignment peaks at intermediate density levels

Abrupt transitions correspond to meaningful semantic changes

Method exposes hierarchical relationships in large corpora

Abstract

Recent advances in large language models enable documents to be represented as dense semantic embeddings, supporting similarity-based operations over large text collections. However, many web-scale systems still rely on flat clustering or predefined taxonomies, limiting insight into hierarchical topic relationships. In this paper we operationalize hierarchical density modeling on large language model embeddings in a way not previously explored. Instead of enforcing a fixed taxonomy or single clustering resolution, the method progressively relaxes local density constraints, revealing how compact semantic groups merge into broader thematic regions. The resulting tree encodes multi-scale semantic organization directly from data, making structural relationships between topics explicit. We evaluate the hierarchies on standard text benchmarks, showing that semantic alignment peaks at intermediate density levels and that abrupt transitions correspond to meaningful changes in semantic resolution. Beyond benchmarks, the approach is applied to large institutional and scientific corpora, exposing dominant fields, cross-disciplinary proximities, and emerging thematic clusters. By framing hierarchical structure as an emergent property of density in embedding spaces, this method provides an interpretable, multi-scale representation of semantic structure suitable for large, evolving text collections.