The Spectrum of Random Inner-product Kernel Matrices

TL;DR

This paper analyzes the eigenvalue distribution of large random kernel matrices formed from Gaussian vectors and a general kernel function, revealing new limiting spectral densities especially for non-smooth kernels.

Contribution

It extends previous work by deriving the spectral density for non-smooth kernel functions in high-dimensional regimes, using a cubic equation involving the Stieltjes transform.

Findings

Spectral density converges to a limit described by a cubic equation.

For smooth kernels, the limit is the Marcenko-Pastur distribution.

Non-smooth kernels lead to a new family of limiting densities.

Abstract

We consider n-by-n matrices whose (i, j)-th entry is f(X_i^T X_j), where X_1, ...,X_n are i.i.d. standard Gaussian random vectors in R^p, and f is a real-valued function. The eigenvalue distribution of these random kernel matrices is studied at the "large p, large n" regime. It is shown that, when p and n go to infinity, p/n = \gamma which is a constant, and f is properly scaled so that Var(f(X_i^T X_j)) is O(p^{-1}), the spectral density converges weakly to a limiting density on R. The limiting density is dictated by a cubic equation involving its Stieltjes transform. While for smooth kernel functions the limiting spectral density has been previously shown to be the Marcenko-Pastur distribution, our analysis is applicable to non-smooth kernel functions, resulting in a new family of limiting densities.