Spectral Analysis Of Weighted Laplacians Arising In Data Clustering

TL;DR

This paper analyzes the spectral properties of weighted graph Laplacians and their continuum limits, providing insights into data clustering and informing parameter choices in learning algorithms.

Contribution

It introduces a three-parameter family of differential operators as limits of graph Laplacians and studies their spectral gaps in nearly separated clusters.

Findings

Spectral gap depends on the three parameters and cluster separation.

Numerical results extend theoretical analysis to more complex data scenarios.

Insights into parameter selection improve clustering and semi-supervised learning algorithms.

Abstract

Graph Laplacians computed from weighted adjacency matrices are widely used to identify geometric structure in data, and clusters in particular; their spectral properties play a central role in a number of unsupervised and semi-supervised learning algorithms. When suitably scaled, graph Laplacians approach limiting continuum operators in the large data limit. Studying these limiting operators, therefore, sheds light on learning algorithms. This paper is devoted to the study of a parameterized family of divergence form elliptic operators that arise as the large data limit of graph Laplacians. The link between a three-parameter family of graph Laplacians and a three-parameter family of differential operators is explained. The spectral properties of these differential operators are analyzed in the situation where the data comprises two nearly separated clusters, in a sense which is made precise. In particular, we investigate how the spectral gap depends on the three parameters entering the graph Laplacian, and on a parameter measuring the size of the perturbation from the perfectly clustered case. Numerical results are presented which exemplify and extend the analysis: the computations study situations in which there are two nearly separated clusters, but which violate the assumptions used in our theory; situations in which more than two clusters are present, also going beyond our theory; and situations which demonstrate the relevance of our studies of differential operators for the understanding of finite data problems via the graph Laplacian. The findings provide insight into parameter choices made in learning algorithms which are based on weighted adjacency matrices; they also provide the basis for analysis of the consistency of various unsupervised and semi-supervised learning algorithms, in the large data limit.