An Improved Nearest Neighbour Classifier

TL;DR

This paper introduces a windowed nearest neighbor classifier (WNN) for image recognition, demonstrating its high accuracy on handwritten digit datasets and outperforming humans and shallow neural networks.

Contribution

The paper presents a novel windowed nearest neighbor classifier inspired by neural network architecture, with a theoretical basis in approximation theory, and shows its effectiveness on challenging image datasets.

Findings

WNN misclassifies only 0.42% of EMNIST images.

WNN outperforms humans and shallow ANNs on EMNIST.

WNN achieves high accuracy with a new parameter calibration approach.

Abstract

A windowed version of the Nearest Neighbour (WNN) classifier for images is described. While its construction is inspired by the architecture of Artificial Neural Networks, the underlying theoretical framework is based on approximation theory. We illustrate WNN on the datasets MNIST and EMNIST of images of handwritten digits. In order to calibrate the parameters of WNN, we first study it on the classical MNIST dataset. We then apply WNN with these parameters to the challenging EMNIST dataset. It is demonstrated that WNN misclassifies 0.42% of the images of EMNIST and therefore significantly outperforms predictions by humans and shallow ANNs that both have more than 1.3% of errors.