Distributed Adaptive Nearest Neighbor Classifier: Algorithm and Theory

TL;DR

This paper introduces a distributed adaptive nearest neighbor classifier that uses a data-driven approach for selecting the number of neighbors, with an early stopping rule to enhance efficiency and finite sample performance, supported by theoretical analysis and empirical validation.

Contribution

It proposes a novel distributed adaptive NN classifier with a data-driven tuning parameter selection and an early stopping rule, improving efficiency and theoretical convergence rates.

Findings

Achieves nearly optimal convergence rate with large sub-sample sizes.

Demonstrates improved finite sample performance through simulations.

Validates effectiveness with real-world data application.

Abstract

When data is of an extraordinarily large size or physically stored in different locations, the distributed nearest neighbor (NN) classifier is an attractive tool for classification. We propose a novel distributed adaptive NN classifier for which the number of nearest neighbors is a tuning parameter stochastically chosen by a data-driven criterion. An early stopping rule is proposed when searching for the optimal tuning parameter, which not only speeds up the computation but also improves the finite sample performance of the proposed Algorithm. Convergence rate of excess risk of the distributed adaptive NN classifier is investigated under various sub-sample size compositions. In particular, we show that when the sub-sample sizes are sufficiently large, the proposed classifier achieves the nearly optimal convergence rate. Effectiveness of the proposed approach is demonstrated through simulation studies as well as an empirical application to a real-world dataset.