Sample complexity of learning Mahalanobis distance metrics

TL;DR

This paper analyzes the sample complexity of supervised metric learning, providing bounds that depend on data structure and demonstrating how regularization improves generalization, validated through experiments.

Contribution

It offers PAC-style sample complexity bounds for metric learning and shows how regularization can adapt to data complexity for better performance.

Findings

Sample complexity scales with representation dimension without assumptions.

Leveraging data structure improves sample complexity rates.

Regularization helps distinguish signal in noisy data.

Abstract

Metric learning seeks a transformation of the feature space that enhances prediction quality for the given task at hand. In this work we provide PAC-style sample complexity rates for supervised metric learning. We give matching lower- and upper-bounds showing that the sample complexity scales with the representation dimension when no assumptions are made about the underlying data distribution. However, by leveraging the structure of the data distribution, we show that one can achieve rates that are fine-tuned to a specific notion of intrinsic complexity for a given dataset. Our analysis reveals that augmenting the metric learning optimization criterion with a simple norm-based regularization can help adapt to a dataset's intrinsic complexity, yielding better generalization. Experiments on benchmark datasets validate our analysis and show that regularizing the metric can help discern the signal even when the data contains high amounts of noise.