Minimum Norm Interpolation via The Local Theory of Banach Spaces: The Role of $2$-Uniform Convexity

TL;DR

This paper investigates the minimum-norm interpolator (MNI) under 2-uniform convexity, providing sharp bias and generalization bounds for linear and nonlinear models with non-Gaussian covariates.

Contribution

It establishes the first sharp bounds for non-Gaussian covariates in linear models with norms not induced by inner products, under 2-uniform convexity.

Findings

Sharp bias bounds for MNI under 2-uniform convexity.

Sharp generalization bounds for $\,\ell_p$-MNI with $p$ in a specific range.

Bounded the MNI bias in overparameterized linear and nonlinear models.

Abstract

The minimum-norm interpolator (MNI) framework has recently attracted considerable attention as a tool for understanding generalization in overparameterized models, such as neural networks. In this work, we study the MNI under a $2$-uniform convexity assumption, which is weaker than requiring the norm to be induced by an inner product, and it typically does not admit a closed-form solution. At a high level, we show that this condition yields an upper bound on the MNI bias in both linear and nonlinear models. We further show that this bound is sharp for overparameterized linear regression when the unit ball of the norm is in isotropic (or John's) position, and the covariates are isotropic, symmetric, i.i.d. sub-Gaussian, such as vectors with i.i.d. Bernoulli entries. Finally, under the same assumption on the covariates, we prove sharp generalization bounds for the $\ell_p$-MNI when $p \in \bigl(1 + C/\log d, 2\bigr]$. To the best of our knowledge, this is the first work to establish sharp bounds for non-Gaussian covariates in linear models when the norm is not induced by an inner product. This work is deeply inspired by classical works on $K$-convexity, and more modern work on the geometry of 2-uniform and isotropic convex bodies.