Bounded-Distortion Metric Learning

TL;DR

This paper introduces bounded-distortion metric learning (BDML), a novel framework for embedding metric spaces with constraints to improve stability and generalization in classification and clustering tasks.

Contribution

It proposes a new BDML framework with an efficient solver, extends it to pseudo-metrics, and provides theoretical analysis linking distortion to stability and generalization.

Findings

BDML achieves promising results on benchmark datasets.

The bounded-distortion constraint improves stability and reduces overfitting.

Theoretical analysis confirms distortion's role in generalization.

Abstract

Metric learning aims to embed one metric space into another to benefit tasks like classification and clustering. Although a greatly distorted metric space has a high degree of freedom to fit training data, it is prone to overfitting and numerical inaccuracy. This paper presents {\it bounded-distortion metric learning} (BDML), a new metric learning framework which amounts to finding an optimal Mahalanobis metric space with a bounded-distortion constraint. An efficient solver based on the multiplicative weights update method is proposed. Moreover, we generalize BDML to pseudo-metric learning and devise the semidefinite relaxation and a randomized algorithm to approximately solve it. We further provide theoretical analysis to show that distortion is a key ingredient for stability and generalization ability of our BDML algorithm. Extensive experiments on several benchmark datasets yield promising results.