Adaptive Kernel Selection for Kernelized Diffusion Maps

TL;DR

This paper introduces adaptive kernel selection methods for Kernelized Diffusion Maps, improving the stability and accuracy of spectral estimations in kernel-based diffusion processes.

Contribution

It presents two novel approaches—differentiable variational optimization and unsupervised cross-validation—for selecting kernels in KDM, backed by theoretical guarantees.

Findings

Lipschitz dependence of KDM operators on kernel weights

Continuity of spectral projectors under a gap condition

Exponential consistency of the cross-validation selector

Abstract

Selecting an appropriate kernel is a central challenge in kernel-based spectral methods. In \emph{Kernelized Diffusion Maps} (KDM), the kernel determines the accuracy of the RKHS estimator of a diffusion-type operator and hence the quality and stability of the recovered eigenfunctions. We introduce two complementary approaches to adaptive kernel selection for KDM. First, we develop a variational outer loop that learns continuous kernel parameters, including bandwidths and mixture weights, by differentiating through the Cholesky-reduced KDM eigenproblem with an objective combining eigenvalue maximization, subspace orthonormality, and RKHS regularization. Second, we propose an unsupervised cross-validation pipeline that selects kernel families and bandwidths using an eigenvalue-sum criterion together with random Fourier features for scalability. Both methods share a common theoretical foundation: we prove Lipschitz dependence of KDM operators on kernel weights, continuity of spectral projectors under a gap condition, a residual-control theorem certifying proximity to the target eigenspace, and exponential consistency of the cross-validation selector over a finite kernel dictionary.