A theory of shape regularity for local regression maps

TL;DR

This paper introduces shape-regular regression maps as a framework to analyze the convergence rates of local regression estimators, providing theoretical bounds and demonstrating their necessity for optimal performance.

Contribution

It develops a theoretical framework using Vapnik-Chervonenkis theory to establish bounds on estimation errors for local regression methods, highlighting the importance of shape regularity.

Findings

Shape regularity is necessary and sufficient for optimal convergence rates.

New concentration bounds for local regression methods like nearest neighbors and regression trees.

The framework applies even when localization depends on the full data sample.

Abstract

We introduce the concept of shape-regular regression maps as a framework to derive optimal rates of convergence for various non-parametric local regression estimators. Using Vapnik-Chervonenkis theory, we establish upper and lower bounds on the pointwise and the sup-norm estimation error, even when the localization procedure depends on the full data sample, and under mild conditions on the regression model. Our results demonstrate that the shape regularity of regression maps is not only sufficient but also necessary to achieve an optimal rate of convergence for Lipschitz regression functions. To illustrate the theory, we establish new concentration bounds for many popular local regression methods such as nearest neighbors algorithm, CART-like regression trees and several purely random trees including Mondrian trees.