Robustness and Generalization for Metric Learning

TL;DR

This paper introduces a robustness-based framework to analyze and derive generalization bounds for metric learning algorithms, establishing weak robustness as necessary and sufficient for generalization.

Contribution

It adapts the concept of algorithmic robustness to metric learning, providing new theoretical insights and bounds for a broad class of algorithms, including sparse formulations.

Findings

Weak robustness is necessary and sufficient for metric learning generalization.

Derived generalization bounds for many existing metric learning algorithms.

Extended analysis to sparse metric learning formulations.

Abstract

Metric learning has attracted a lot of interest over the last decade, but the generalization ability of such methods has not been thoroughly studied. In this paper, we introduce an adaptation of the notion of algorithmic robustness (previously introduced by Xu and Mannor) that can be used to derive generalization bounds for metric learning. We further show that a weak notion of robustness is in fact a necessary and sufficient condition for a metric learning algorithm to generalize. To illustrate the applicability of the proposed framework, we derive generalization results for a large family of existing metric learning algorithms, including some sparse formulations that are not covered by previous results.