On the optimal shape parameter for kernel methods: Sharp direct and inverse statements

TL;DR

This paper develops a theoretical framework to determine the optimal shape parameter for RBF kernels, linking it to superconvergence phenomena and covering practical kernels used in machine learning.

Contribution

It introduces a novel theoretical approach connecting shape parameter optimization with superconvergence, applicable to a broad class of Sobolev kernels.

Findings

Established a link between shape parameter and superconvergence.

Provided sharp direct and inverse approximation statements.

Clarified the interaction between kernel regularity and parameter choice.

Abstract

The search for the optimal shape parameter for Radial Basis Function (RBF) kernel approximation has been an outstanding research problem for decades. In this work, we establish a theoretical framework for this problem by leveraging a recently established theory on sharp direct, inverse and saturation statements for kernel based approximation. In particular, we link the search for the optimal shape parameter to superconvergence phenomena. Our analysis is carried out for finitely smooth Sobolev kernels, thereby covering large classes of radial kernels used in practice, including those emerging from current machine-learning methodologies. Our results elucidate how approximation regimes, kernel regularity, and parameter choices interact, thereby clarifying a question that has remained unresolved for decades.