Sample-based almost-sure quasi-optimal approximation in reproducing kernel Hilbert spaces

TL;DR

This paper develops a probabilistic sampling method for function approximation in RKHS that achieves near-optimal accuracy with fewer samples, introducing novel sampling schemes and theoretical guarantees.

Contribution

It introduces subspace-informed volume sampling and a greedy subsampling scheme for efficient, quasi-optimal function approximation in RKHS, with theoretical analysis and numerical validation.

Findings

Random Christoffel sampling achieves controllable quasi-optimality with high probability.

Subspace-informed volume sampling outperforms classical methods in numerical experiments.

Greedy subsampling reduces sample size while maintaining approximation quality.

Abstract

This paper addresses the problem of approximating an unknown function from point evaluations. When obtaining these point evaluations is costly, minimising the required sample size becomes crucial, and it is unreasonable to reserve a sufficiently large test sample for estimating the approximation accuracy. Therefore, an approximation with a certified quasi-optimality factor is required. This article shows that such an approximation can be obtained when the sought function lies in a reproducing kernel Hilbert space (RKHS) and is to be approximated in a finite-dimensional linear subspace $\mathcal{V}_d$. However, selecting the sample points to minimise the quasi-optimality factor requires optimising over an infinite set of points and computing exact inner products in RKHS, which is often infeasible in practice. Extending results from optimal sampling for $L^2$ approximation, the present paper proves that random points, drawn independently from the Christoffel sampling distribution associated with $\mathcal{V}_d$, can yield a controllable quasi-optimality factor with high probability. Inspired by this result, a novel sampling scheme, coined subspace-informed volume sampling, is introduced and evaluated in numerical experiments, where it outperforms classical i.i.d. Christoffel sampling and continuous volume sampling. To reduce the size of such a random sample, an additional greedy subsampling scheme with provable suboptimality bounds is introduced. Our presentation is of independent interest to the inverse problems community, as it offers a simpler interpretation of the parametrised background data weak (PBDW) method.