Debiasing Polynomial and Fourier Regression

TL;DR

This paper introduces a debiasing technique for polynomial and Fourier regression that achieves unbiased estimates with near-optimal sample complexity, improving upon existing biased methods through a novel connection with random matrix theory.

Contribution

We develop a simple debiasing method for polynomial regression using eigenvalues of random matrices, providing unbiased estimates and extending to Fourier series approximation with optimal sample complexity.

Findings

Our estimator is unbiased and sample-efficient.

Experimental results show improved performance over iid leverage score sampling.

Techniques enable debiasing of Fourier series approximation methods.

Abstract

We study the problem of approximating an unknown function $f:\mathbb{R}\to\mathbb{R}$ by a degree-$d$ polynomial using as few function evaluations as possible, where error is measured with respect to a probability distribution $\mu$. Existing randomized algorithms achieve near-optimal sample complexities to recover a $ (1+\varepsilon) $-optimal polynomial but produce biased estimates of the best polynomial approximation, which is undesirable. We propose a simple debiasing method based on a connection between polynomial regression and random matrix theory. Our method involves evaluating $f(\lambda_1),\ldots,f(\lambda_{d+1})$ where $\lambda_1,\ldots,\lambda_{d+1}$ are the eigenvalues of a suitably designed random complex matrix tailored to the distribution $\mu$. Our estimator is unbiased, has near-optimal sample complexity, and experimentally outperforms iid leverage score sampling. Additionally, our techniques enable us to debias existing methods for approximating a periodic function with a truncated Fourier series with near-optimal sample complexity.