Generalization error of minimum weighted norm and kernel interpolation

TL;DR

This paper analyzes how minimum weighted norm and kernel interpolation methods generalize as model complexity increases, revealing their implicit bias and convergence properties through a deterministic approximation theory approach.

Contribution

It provides a rigorous theoretical framework for understanding the implicit bias and generalization of weighted norm interpolants, including special cases like trigonometric polynomials and spherical harmonics.

Findings

Interpolants converge to a RKHS as parameters grow

Generalization errors diminish with increasing parameters

The implicit bias explains over-parameterization effects

Abstract

We study the generalization error of functions that interpolate prescribed data points and are selected by minimizing a weighted norm. Under natural and general conditions, we prove that both the interpolants and their generalization errors converge as the number of parameters grow, and the limiting interpolant belongs to a reproducing kernel Hilbert space. This rigorously establishes an implicit bias of minimum weighted norm interpolation and explains why norm minimization may either benefit or suffer from over-parameterization. As special cases of this theory, we study interpolation by trigonometric polynomials and spherical harmonics. Our approach is from a deterministic and approximation theory viewpoint, as opposed to a statistical or random matrix one.