Feasibility Based Large Margin Nearest Neighbor Metric Learning

TL;DR

This paper improves the Large Margin Nearest Neighbor (LMNN) metric learning algorithm by analyzing its feasibility constraints and introducing a weighting scheme that enhances the likelihood of finding a better metric, leading to improved classification accuracy.

Contribution

It introduces a feasibility measure and a weighting scheme to enhance LMNN's optimization process, resulting in better metrics and accuracy.

Findings

Improved accuracy on synthetic datasets

Enhanced performance on real datasets

Feasibility-based weighting increases success rate

Abstract

Large margin nearest neighbor (LMNN) is a metric learner which optimizes the performance of the popular $k$NN classifier. However, its resulting metric relies on pre-selected target neighbors. In this paper, we address the feasibility of LMNN's optimization constraints regarding these target points, and introduce a mathematical measure to evaluate the size of the feasible region of the optimization problem. We enhance the optimization framework of LMNN by a weighting scheme which prefers data triplets which yield a larger feasible region. This increases the chances to obtain a good metric as the solution of LMNN's problem. We evaluate the performance of the resulting feasibility-based LMNN algorithm using synthetic and real datasets. The empirical results show an improved accuracy for different types of datasets in comparison to regular LMNN.