Incremental Graph Construction Enables Robust Spectral Clustering of Texts

TL;DR

This paper presents an incremental $k$-NN graph construction method that guarantees connectivity, improving spectral clustering robustness on text embeddings, especially at low sparsity levels.

Contribution

The authors introduce a simple incremental $k$-NN graph construction technique that ensures connectivity by design, enhancing spectral clustering stability on text datasets.

Findings

Outperforms standard $k$-NN graphs at low $k$ values.

Maintains comparable performance to standard $k$-NN at higher $k$.

Ensures connected graphs for any $k$ through incremental construction.

Abstract

Neighborhood graphs are a critical but often fragile step in spectral clustering of text embeddings. On realistic text datasets, standard $k$-NN graphs can contain many disconnected components at practical sparsity levels (small $k$), making spectral clustering degenerate and sensitive to hyperparameters. We introduce a simple incremental $k$-NN graph construction that preserves connectivity by design: each new node is linked to its $k$ nearest previously inserted nodes, which guarantees a connected graph for any $k$. We provide an inductive proof of connectedness and discuss implications for incremental updates when new documents arrive. We validate the approach on spectral clustering of SentenceTransformer embeddings using Laplacian eigenmaps across six clustering datasets from the Massive Text Embedding Benchmark. Compared to standard $k$-NN graphs, our method outperforms in the low-$k$ regime where disconnected components are prevalent, and matches standard $k$-NN at larger $k$.