Efficient Function Approximation Under Heteroskedastic Noise

TL;DR

This paper introduces HeteroChebtrunc, an efficient algorithm for function approximation with heteroskedastic noise, achieving tighter error bounds and faster computation than previous methods, along with a new variance estimator bound.

Contribution

The paper presents HeteroChebtrunc, a novel $O(N+ ilde{N}\, ext{log} ilde{N})$ algorithm for heteroskedastic noise, improving approximation accuracy and computational efficiency.

Findings

HeteroChebtrunc outperforms NoisyChebtrunc in error bounds.

The algorithm operates in near-linear time with respect to sample size.

A new high-probability variance estimator bound is derived.

Abstract

Approximating a function $f(x)$ on $[-1,1]$ based on $N+1$ samples is a classical problem in numerical analysis. If the samples come with heteroskedastic noise depending on $x$ of variance $\sigma(x)^2$, an $O(N\log N)$ algorithm for this problem has not yet been found in the current literature. In this paper, we propose a method called HeteroChebtrunc, adapted from an algorithm named NoisyChebtrunc. Using techniques in high-dimensional probability, we show that with high probability, HeteroChebtrunc achieves a tighter infinity-norm error bound than NoisyChebtrunc under heteroskedastic noise. This algorithm runs in $O(N+\hat{N}\log \hat{N})$ operations, where $\hat{N}\ll N$ is a chosen parameter. While investigating the properties of HeteroChebtrunc, we also derive a high-probability non-asymptotic relative error bound on the sample variance estimator for subgaussian variables, which is potentially another result of broader interest. We provide numerical experiments to demonstrate the improved uniform error of our algorithm.