Function Approximation in Numerically Rank-Deficient Bases

TL;DR

This paper analyzes how finite-precision arithmetic affects linear function approximation in non-orthogonal bases, revealing the importance of regularization and sampling strategies to mitigate numerical rank-deficiency effects.

Contribution

It introduces the concept of numerical span, provides a model for rounding errors, and demonstrates how regularization improves approximation stability and sampling efficiency in finite-precision settings.

Findings

Approximation behaves like exact arithmetic with an additional penalty due to rounding.

Regularization controls error amplification and relaxes sampling conditions.

Regularized Christoffel function is computable and improves sampling bounds.

Abstract

We study linear function approximation in a finite basis under finite-precision arithmetic. In a highly non-orthogonal basis, certain directions are only weakly represented, so that rounding errors can significantly distort the effectively spanned space. In the first part of the paper, we formalize this phenomenon through the notion of a numerical span. Using a novel model for the rounding errors involved, we prove that approximation in the numerical span behaves like approximation in exact arithmetic subject to an additional penalty proportional to the size of the expansion coefficients and the unit roundoff. A key implication is that straightforward numerical orthogonalization cannot mitigate the effects induced by finite-precision arithmetic. The framework also provides a theoretical justification for $\ell^2$-regularized approximation. Moreover, regularization controls the amplification of rounding errors in the computation of expansion coefficients. In the second part of the paper, we address sampling for function approximation in the presence of numerical rank-deficiency. We demonstrate that regularization has another fundamental benefit: it relaxes the conditions required for accurate least squares approximation from sampled data. This effect is made concrete through an analysis of randomized sampling based on a regularized variant of the Christoffel function. The resulting sample complexity bounds depend on an effective dimension that measures the number of directions that remain useful after finite-precision rounding. We also show that regularization renders the Christoffel function computable in contrast to the standard Christoffel function, whose numerical evaluation may require arbitrarily high precision in the presence of numerical rank-deficiency. We apply the derived theory to obtain new results for the discretization of univariate Fourier extension frames.