Random Forests for Metric Learning with Implicit Pairwise Position Dependence

TL;DR

This paper introduces a novel metric learning approach using random forests to implicitly adapt the distance function across the feature space, improving accuracy and efficiency over existing methods.

Contribution

It proposes a single, adaptive metric learning method based on random forests that captures both absolute and relative position information.

Findings

Outperforms state-of-the-art global and multi-metric methods in accuracy.

Achieves up to 16x faster computation than multi-metric methods.

Consistently ranked first in accuracy across various datasets.

Abstract

Metric learning makes it plausible to learn distances for complex distributions of data from labeled data. However, to date, most metric learning methods are based on a single Mahalanobis metric, which cannot handle heterogeneous data well. Those that learn multiple metrics throughout the space have demonstrated superior accuracy, but at the cost of computational efficiency. Here, we take a new angle to the metric learning problem and learn a single metric that is able to implicitly adapt its distance function throughout the feature space. This metric adaptation is accomplished by using a random forest-based classifier to underpin the distance function and incorporate both absolute pairwise position and standard relative position into the representation. We have implemented and tested our method against state of the art global and multi-metric methods on a variety of data sets. Overall, the proposed method outperforms both types of methods in terms of accuracy (consistently ranked first) and is an order of magnitude faster than state of the art multi-metric methods (16x faster in the worst case).