Achieving the time of $1$-NN, but the accuracy of $k$-NN

TL;DR

This paper introduces a distributed method that combines denoised 1-NN predictors over small subsamples to nearly match the accuracy of k-NN while maintaining the fast prediction time of 1-NN, suitable for large-scale data.

Contribution

The paper presents a novel distributed aggregation approach that achieves k-NN accuracy with 1-NN prediction speed, supported by theoretical analysis and empirical validation.

Findings

Small subsample sizes suffice for k-NN-like accuracy

The method maintains the computational efficiency of 1-NN

The approach is effective both theoretically and experimentally

Abstract

We propose a simple approach which, given distributed computing resources, can nearly achieve the accuracy of $k$-NN prediction, while matching (or improving) the faster prediction time of $1$-NN. The approach consists of aggregating denoised $1$-NN predictors over a small number of distributed subsamples. We show, both theoretically and experimentally, that small subsample sizes suffice to attain similar performance as $k$-NN, without sacrificing the computational efficiency of $1$-NN.