Data-driven kernel designs for optimized greedy schemes: A machine learning perspective

TL;DR

This paper introduces a machine learning-inspired approach to optimize kernel functions for meshfree methods, enhancing their accuracy and robustness through data-driven hyperparameter tuning and greedy algorithms.

Contribution

It proposes two-layered kernel machines that generalize classical RBFs by optimizing multiple hyperparameters, including kernel rotations, and integrates greedy algorithms for adaptive basis construction.

Findings

Kernel optimization improves interpolation accuracy.

Two-layered kernels outperform single hyperparameter models.

Greedy algorithms effectively adapt basis functions to data.

Abstract

Thanks to their easy implementation via Radial Basis Functions (RBFs), meshfree kernel methods have been proved to be an effective tool for e.g. scattered data interpolation, PDE collocation, classification and regression tasks. Their accuracy might depend on a length scale hyperparameter, which is often tuned via cross validation schemes. Here we leverage approaches and tools from the machine learning community to introduce two-layered kernel machines, which generalize the classical RBF approaches that rely on a single hyperparameter. Indeed, the proposed learning strategy returns a kernel that is optimized not only in the Euclidean directions, but that further incorporates kernel rotations. The kernel optimization is shown to be robust by using recently improved calculations of cross validation scores. Finally, the use of greedy approaches, and specifically of the Vectorial Kernel Orthogonal Greedy Algorithm (VKOGA), allows us to construct an optimized basis that adapts to the data. Beyond a rigorous analysis on the convergence of the so-constructed two-Layered (2L)-VKOGA, its benefits are highlighted on both synthesized and real benchmark data sets.