Regularity based spectral clustering and mapping the Fiedler-carpet

TL;DR

This paper explores spectral clustering through singular value decomposition and normalized Laplacian methods, revealing a spectral gap phenomenon and introducing the concept of the Fiedler-carpet for graph partitioning.

Contribution

It extends spectral clustering to rectangular arrays and directed graphs, and introduces the Fiedler-carpet concept linking eigenvalues to cluster structure.

Findings

Spectral gap indicates a sudden decrease in cluster variance at specific cluster counts.

The Fiedler-carpet is formed by the first $k-1$ eigenvectors, capturing essential clustering information.

Application to directed migration graphs demonstrates practical relevance.

Abstract

Spectral clustering is discussed from many perspectives, by extending it to rectangular arrays and discrepancy minimization too. Near optimal clusters are obtained with singular value decomposition and with the weighted $k$-means algorithm. In case of rectangular arrays, this means enhancing the method of correspondence analysis with clustering, and in case of edge-weighted graphs, a normalized Laplacian based clustering. In the latter case it is proved that a spectral gap between the $(k-1)$th and $k$th smallest positive eigenvalues of the normalized Laplacian matrix gives rise to a sudden decrease of the inner cluster variances when the number of clusters of the vertex representatives is $2^{k-1}$, but only the first $k-1$ eigenvectors, constituting the so-called Fiedler-carpet, are used in the representation. Application to directed migration graphs is also discussed.