On the stability and accuracy of least squares approximations

TL;DR

This paper establishes a criterion for the regularization needed in least squares approximations to ensure stability and near-best accuracy when reconstructing functions from random or deterministic samples.

Contribution

It provides a new stability criterion for least squares approximations that balances approximation accuracy and stability, applicable to various approximation schemes.

Findings

Derived a stability criterion for least squares methods

Validated the criterion for random and deterministic sampling schemes

Applied results to polynomial and trigonometric approximation schemes

Abstract

We consider the problem of reconstructing an unknown function $f$ on a domain $X$ from samples of $f$ at $n$ randomly chosen points with respect to a given measure $\rho_X$. Given a sequence of linear spaces $(V_m)_{m>0}$ with ${\rm dim}(V_m)=m\leq n$, we study the least squares approximations from the spaces $V_m$. It is well known that such approximations can be inaccurate when $m$ is too close to $n$, even when the samples are noiseless. Our main result provides a criterion on $m$ that describes the needed amount of regularization to ensure that the least squares method is stable and that its accuracy, measured in $L^2(X,\rho_X)$, is comparable to the best approximation error of $f$ by elements from $V_m$. We illustrate this criterion for various approximation schemes, such as trigonometric polynomials, with $\rho_X$ being the uniform measure, and algebraic polynomials, with $\rho_X$ being either the uniform or Chebyshev measure. For such examples we also prove similar stability results using deterministic samples that are equispaced with respect to these measures.