Rates of Convergence for Nearest Neighbor Classification

TL;DR

This paper analyzes the convergence rates of nearest neighbor classifiers in metric spaces, providing finite-sample bounds and broadening the understanding of their consistency across various data spaces.

Contribution

It offers the first finite-sample, distribution-dependent convergence rates for nearest neighbor classifiers under minimal assumptions, extending their known universal consistency.

Findings

Established finite-sample convergence rates.

Proved universal consistency in broader data spaces.

Introduced smoothness classes tailored for nearest neighbor methods.

Abstract

Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates for nearest neighbor classification has not fully reflected these subtle properties. We analyze the behavior of these estimators in metric spaces and provide finite-sample, distribution-dependent rates of convergence under minimal assumptions. As a by-product, we are able to establish the universal consistency of nearest neighbor in a broader range of data spaces than was previously known. We illustrate our upper and lower bounds by introducing smoothness classes that are customized for nearest neighbor classification.