Spectral Properties of Radial Kernels and Clustering in High Dimensions

TL;DR

This paper analyzes the spectral properties of radial kernels in high-dimensional mixture models, demonstrating how kernel PCA can effectively cluster such data even with minimal angular separation between covariances.

Contribution

It provides new theoretical insights into the eigenstructure of radial kernels in high dimensions and introduces a spectral clustering algorithm with minimal separation requirements.

Findings

Kernel matrices exhibit specific spectral structures in high dimensions.

Kernel PCA can successfully cluster mixtures with small angular separation.

The proposed spectral algorithm is effective even when covariance matrices are nearly aligned.

Abstract

In this paper, we study the spectrum and the eigenvectors of radial kernels for mixtures of distributions in $\mathbb{R}^n$. Our approach focuses on high dimensions and relies solely on the concentration properties of the components in the mixture. We give several results describing of the structure of kernel matrices for a sample drawn from such a mixture. Based on these results, we analyze the ability of kernel PCA to cluster high dimensional mixtures. In particular, we exhibit a specific kernel leading to a simple spectral algorithm for clustering mixtures with possibly common means but different covariance matrices. We show that the minimum angular separation between the covariance matrices that is required for the algorithm to succeed tends to $0$ as $n$ goes to infinity.