An Affine Invariant $k$-Nearest Neighbor Regression Estimate

TL;DR

This paper introduces an affine-invariant metric for $k$-nearest neighbor regression, ensuring asymptotic consistency under minimal data assumptions and affine transformations.

Contribution

It proposes a novel data-dependent metric that makes the $k$-NN regression affine-invariant and proves its asymptotic consistency.

Findings

The new metric is invariant under all affine transformations.

The $k$-NN regression with this metric is asymptotically consistent.

Minimal data assumptions are sufficient for consistency.

Abstract

We design a data-dependent metric in $\mathbb R^d$ and use it to define the $k$-nearest neighbors of a given point. Our metric is invariant under all affine transformations. We show that, with this metric, the standard $k$-nearest neighbor regression estimate is asymptotically consistent under the usual conditions on $k$, and minimal requirements on the input data.