Discriminative Learning of the Prototype Set for Nearest Neighbor Classification

TL;DR

This paper introduces a novel dissimilarity-based, parametrized model for nearest neighbor classification that optimizes prototype selection through a large-margin principle, enhancing efficiency and generalization.

Contribution

It proposes a new model that learns prototype parameters to improve nearest neighbor classification without relying on input vector space assumptions.

Findings

Model reduces violation of nearest neighbor rule on training data

Empirical comparisons show superior performance over existing methods

Formulation enables tractable optimization using numerical techniques

Abstract

The nearest neighbor rule is a classic yet essential classification model, particularly in problems where the supervising information is given by pairwise dissimilarities and the embedding function are not easily obtained. Prototype selection provides means of generalization and improving efficiency of the nearest neighbor model, but many existing methods assume and rely on the analyses of the input vector space. In this paper, we explore a dissimilarity-based, parametrized model of the nearest neighbor rule. In the proposed model, the selection of the nearest prototypes is influenced by the parameters of the respective prototypes. It provides a formulation for minimizing the violation of the extended nearest neighbor rule over the training set in a tractable form to exploit numerical techniques. We show that the minimization problem reduces to a large-margin principle learning and demonstrate its advantage by empirical comparisons with other prototype selection methods.