On high-dimensional modifications of the nearest neighbor classifier

TL;DR

This paper examines the limitations of the nearest neighbor classifier in high-dimensional, low-sample size settings and proposes new modifications, supported by theoretical analysis and empirical comparisons.

Contribution

It introduces novel high-dimensional modifications to the nearest neighbor classifier and provides theoretical and empirical evaluations of their effectiveness.

Findings

Proposed methods improve classification accuracy in HDLSS scenarios.

Theoretical analysis explains the behavior of modifications in high dimensions.

Empirical results demonstrate superior performance over existing approaches.

Abstract

Nearest neighbor classifier is arguably the most simple and popular nonparametric classifier available in the literature. However, due to the concentration of pairwise distances and the violation of the neighborhood structure, this classifier often suffers in high-dimension, low-sample size (HDLSS) situations, especially when the scale difference between the competing classes dominates their location difference. Several attempts have been made in the literature to take care of this problem. In this article, we discuss some of these existing methods and propose some new ones. We carry out some theoretical investigations in this regard and analyze several simulated and benchmark datasets to compare the empirical performances of proposed methods with some of the existing ones.