Consistency of Anchor-based Spectral Clustering

TL;DR

This paper introduces a theoretically grounded anchor-based spectral clustering algorithm that is consistent in the asymptotic limit, demonstrating practical effectiveness and advantages over standard methods on synthetic and real datasets.

Contribution

The paper provides the first rigorous theoretical analysis of an anchor-based spectral clustering method, establishing its consistency and practical parameter guidelines.

Findings

The algorithm is consistent and can recover disjoint clusters with high probability.

It performs better than standard spectral clustering in experiments.

It is competitive with state-of-the-art large-scale clustering methods.

Abstract

Anchor-based techniques reduce the computational complexity of spectral clustering algorithms. Although empirical tests have shown promising results, there is currently a lack of theoretical support for the anchoring approach. We define a specific anchor-based algorithm and show that it is amenable to rigorous analysis, as well as being effective in practice. We establish the theoretical consistency of the method in an asymptotic setting where data is sampled from an underlying continuous probability distribution. In particular, we provide sharp asymptotic conditions for the algorithm parameters which ensure that the anchor-based method can recover with high probability disjoint clusters that are mutually separated by a positive distance. We illustrate the performance of the algorithm on synthetic data and explain how the theoretical convergence analysis can be used to inform the practical choice of parameter scalings. We also test the accuracy and efficiency of the algorithm on two large scale real data sets. We find that the algorithm offers clear advantages over standard spectral clustering. We also find that it is competitive with the state-of-the-art LSC method of Chen and Cai (Twenty-Fifth AAAI Conference on Artificial Intelligence, 2011), while having the added benefit of a consistency guarantee.