Optimal 1-NN Prototypes for Pathological Geometries

TL;DR

This paper addresses the challenge of selecting optimal prototypes for 1-NN classifiers in complex geometries, providing an analytical solution and an algorithm for nearly-optimal prototypes, validated through empirical experiments.

Contribution

It introduces an analytical method for finding optimal prototypes in difficult geometries and proposes a new algorithm for near-optimal prototype selection.

Findings

Analytical solutions for optimal prototypes in challenging geometries.

Proposed algorithm effectively finds nearly-optimal prototypes.

Empirical validation confirms theoretical results.

Abstract

Using prototype methods to reduce the size of training datasets can drastically reduce the computational cost of classification with instance-based learning algorithms like the k-Nearest Neighbour classifier. The number and distribution of prototypes required for the classifier to match its original performance is intimately related to the geometry of the training data. As a result, it is often difficult to find the optimal prototypes for a given dataset, and heuristic algorithms are used instead. However, we consider a particularly challenging setting where commonly used heuristic algorithms fail to find suitable prototypes and show that the optimal prototypes can instead be found analytically. We also propose an algorithm for finding nearly-optimal prototypes in this setting, and use it to empirically validate the theoretical results.