Uniform convergence for Gaussian kernel ridge regression

TL;DR

This paper proves the first polynomial convergence rates for Gaussian kernel ridge regression in both uniform and L2 norms, enhancing theoretical understanding of its performance with fixed hyperparameters.

Contribution

It establishes the first polynomial convergence rates for Gaussian KRR in uniform and L2 norms with fixed hyperparameters, filling a key gap in theory.

Findings

Polynomial uniform convergence rates are established.

Polynomial L2 convergence rates are proved for fixed kernel width.

Results justify the use of Gaussian KRR with fixed hyperparameters.

Abstract

This paper establishes the first polynomial convergence rates for Gaussian kernel ridge regression (KRR) with a fixed hyperparameter in both the uniform and the $L^{2}$-norm. The uniform convergence result closes a gap in the theoretical understanding of KRR with the Gaussian kernel, where no such rates were previously known. In addition, we prove a polynomial $L^{2}$-convergence rate in the case, where the Gaussian kernel's width parameter is fixed. This also contributes to the broader understanding of smooth kernels, for which previously only sub-polynomial $L^{2}$-rates were known in similar settings. Together, these results provide new theoretical justification for the use of Gaussian KRR with fixed hyperparameters in nonparametric regression.