A new approach to locally adaptive polynomial regression

TL;DR

This paper presents a novel bandwidth selection method for local polynomial regression that adaptively adjusts to the function's smoothness, providing near-optimal risk bounds and a new perspective on nonparametric regression.

Contribution

Introduces a new bandwidth selection procedure based on $ ext{L}_2$-norm criteria, offering a distinct approach to local adaptivity in polynomial regression.

Findings

Achieves non-asymptotic risk bounds with local adaptivity

Identifies a single global tuning parameter for optimal performance

Develops a new bandwidth selection equation of independent interest

Abstract

Adaptive bandwidth selection is a fundamental challenge in nonparametric regression. This paper introduces a new bandwidth selection procedure inspired by the optimality criteria for $\ell_0$-penalized regression. Although similar in spirit to Lepski's method and its variants in selecting the largest interval satisfying an admissibility criterion, our approach stems from a distinct philosophy, utilizing criteria based on $\ell_2$-norms of interval projections rather than explicit point and variance estimates. We obtain non-asymptotic risk bounds for the local polynomial regression methods based on our bandwidth selection procedure which adapt (near-)optimally to the local H\"{o}lder exponent of the underlying regression function simultaneously at all points in its domain. Furthermore, we show that there is a single ideal choice of a global tuning parameter in each case under which the above-mentioned local adaptivity holds. The optimal risks of our methods derive from the properties of solutions to a new ``bandwidth selection equation'' which is of independent interest. We believe that the principles underlying our approach provide a new perspective to the classical yet ever relevant problem of locally adaptive nonparametric regression.