Analysis of target data-dependent greedy kernel algorithms: Convergence rates for $f$-, $f \cdot P$- and $f/P$-greedy

TL;DR

This paper develops a unified theoretical framework for data-dependent greedy kernel algorithms, proving they can achieve faster convergence rates than data-independent methods without special assumptions on the target function.

Contribution

It introduces a new scale of greedy algorithms with a dependency parameter, providing the first convergence rates showing data-dependent strategies outperform uniform points.

Findings

Target data-dependent algorithms have faster convergence rates.

Convergence rates are confirmed by multiple examples.

Rates are achieved without assumptions on the target function.

Abstract

Data-dependent greedy algorithms in kernel spaces are known to provide fast converging interpolants, while being extremely easy to implement and efficient to run. Despite this experimental evidence, no detailed theory has yet been presented. This situation is unsatisfactory especially when compared to the case of the data-independent $P$-greedy algorithm, for which optimal convergence rates are available, despite its performances being usually inferior to the ones of target data-dependent algorithms. In this work we fill this gap by first defining a new scale of greedy algorithms for interpolation that comprises all the existing ones in a unique analysis, where the degree of dependency of the selection criterion on the functional data is quantified by a real parameter. We then prove new convergence rates where this degree is taken into account and we show that, possibly up to a logarithmic factor, target data-dependent selection strategies provide faster convergence. In particular, for the first time we obtain convergence rates for target data adaptive interpolation that are faster than the ones given by uniform points, without the need of any special assumption on the target function. The rates are confirmed by a number of examples. These results are made possible by a new analysis of greedy algorithms in general Hilbert spaces.