Robust nearest-neighbor methods for classifying high-dimensional data

TL;DR

This paper introduces a robust nearest-neighbor classifier for high-dimensional data that uses thresholding and truncation to improve performance under heavy-tailed distributions, requiring minimal data for effective classification.

Contribution

The paper presents a novel robust nearest-neighbor method that maintains classification boundaries similar to traditional approaches without needing finite moments, even with minimal training data.

Findings

The method performs comparably to traditional classifiers under heavy-tailed data.

It requires only one data vector per population for training.

The approach is robust to correlation and heterogeneity among data components.

Abstract

We suggest a robust nearest-neighbor approach to classifying high-dimensional data. The method enhances sensitivity by employing a threshold and truncates to a sequence of zeros and ones in order to reduce the deleterious impact of heavy-tailed data. Empirical rules are suggested for choosing the threshold. They require the bare minimum of data; only one data vector is needed from each population. Theoretical and numerical aspects of performance are explored, paying particular attention to the impacts of correlation and heterogeneity among data components. On the theoretical side, it is shown that our truncated, thresholded, nearest-neighbor classifier enjoys the same classification boundary as more conventional, nonrobust approaches, which require finite moments in order to achieve good performance. In particular, the greater robustness of our approach does not come at the price of reduced effectiveness. Moreover, when both training sample sizes equal 1, our new method can have performance equal to that of optimal classifiers that require independent and identically distributed data with known marginal distributions; yet, our classifier does not itself need conditions of this type.