Convergence rates of vector-valued local polynomial regression

TL;DR

This paper extends classical convergence rate results of local polynomial regression to high-dimensional target spaces, showing optimal rates are achievable under certain noise assumptions and linking failure probability to sample size.

Contribution

It proves that optimal convergence rates for local polynomial regression also hold when estimating functions into high-dimensional spaces, extending Stone's 1980 results.

Findings

Optimal convergence rates are achievable in high-dimensional target spaces.

A connection between failure probability and sample size is established.

Results depend on assumptions about the noise distribution.

Abstract

Non-parametric estimation of functions as well as their derivatives by means of local-polynomial regression is a subject that was studied in the literature since the late 1970's. Given a set of noisy samples of a $\mathcal{C}^k$ smooth function, we perform a local polynomial fit, and by taking its $m$-th derivative we obtain an estimate for the $m$-th function derivative. The known optimal rates of convergence for this problem for a $k$-times smooth function $f:\mathbb{R}^d \to \mathbb{R}$ are $n^{-\frac{k-m}{2k + d}}$. However in modern applications it is often the case that we have to estimate a function operating to $\mathbb{R}^D$, for $D \gg d$ extremely large. In this work, we prove that these same rates of convergence are also achievable by local-polynomial regression in case of a high dimensional target, given some assumptions on the noise distribution. This result is an extension to Stone's seminal work from 1980 to the regime of high-dimensional target domain. In addition, we unveil a connection between the failure probability $\varepsilon$ and the number of samples required to achieve the optimal rates.