Consistency of spectral clustering

TL;DR

This paper investigates the statistical consistency of spectral clustering algorithms, demonstrating that normalized spectral clustering reliably converges with increasing data, unlike unnormalized clustering which requires strict conditions.

Contribution

The paper develops new methods to prove the convergence of spectral clustering eigenvectors and compares normalized and unnormalized approaches under general conditions.

Findings

Normalized spectral clustering is consistent under broad conditions.

Unnormalized spectral clustering requires strong additional assumptions.

Normalized spectral clustering is generally superior in real data scenarios.

Abstract

Consistency is a key property of all statistical procedures analyzing randomly sampled data. Surprisingly, despite decades of work, little is known about consistency of most clustering algorithms. In this paper we investigate consistency of the popular family of spectral clustering algorithms, which clusters the data with the help of eigenvectors of graph Laplacian matrices. We develop new methods to establish that, for increasing sample size, those eigenvectors converge to the eigenvectors of certain limit operators. As a result, we can prove that one of the two major classes of spectral clustering (normalized clustering) converges under very general conditions, while the other (unnormalized clustering) is only consistent under strong additional assumptions, which are not always satisfied in real data. We conclude that our analysis provides strong evidence for the superiority of normalized spectral clustering.