New Probabilistic Bounds on Eigenvalues and Eigenvectors of Random Kernel Matrices

TL;DR

This paper improves concentration bounds for eigenvalues and eigenvectors of kernel matrices, especially for radial basis functions, enhancing understanding of spectral properties in kernel methods.

Contribution

It provides new, sharper concentration bounds for kernel matrix eigenvalues, addressing limitations of previous results and including a case study on kernel target-alignment.

Findings

Enhanced bounds for eigenvalues of kernel matrices

Characterization of bounds using sample covariance eigenvalues

Derived concentration inequality for kernel target-alignment

Abstract

Kernel methods are successful approaches for different machine learning problems. This success is mainly rooted in using feature maps and kernel matrices. Some methods rely on the eigenvalues/eigenvectors of the kernel matrix, while for other methods the spectral information can be used to estimate the excess risk. An important question remains on how close the sample eigenvalues/eigenvectors are to the population values. In this paper, we improve earlier results on concentration bounds for eigenvalues of general kernel matrices. For distance and inner product kernel functions, e.g. radial basis functions, we provide new concentration bounds, which are characterized by the eigenvalues of the sample covariance matrix. Meanwhile, the obstacles for sharper bounds are accounted for and partially addressed. As a case study, we derive a concentration inequality for sample kernel target-alignment.