Towards Certified Robustness of Distance Metric Learning

TL;DR

This paper introduces an adversarial margin concept in input space for metric learning, proposing a novel loss to improve robustness and generalization, validated by experiments on multiple datasets.

Contribution

It proposes a new adversarial margin in input space and a perturbation loss to enhance robustness and generalization in metric learning.

Findings

Improved discrimination accuracy over state-of-the-art methods.

Enhanced robustness against input noise.

Theoretical validation of generalization benefits.

Abstract

Metric learning aims to learn a distance metric such that semantically similar instances are pulled together while dissimilar instances are pushed away. Many existing methods consider maximizing or at least constraining a distance margin in the feature space that separates similar and dissimilar pairs of instances to guarantee their generalization ability. In this paper, we advocate imposing an adversarial margin in the input space so as to improve the generalization and robustness of metric learning algorithms. We first show that, the adversarial margin, defined as the distance between training instances and their closest adversarial examples in the input space, takes account of both the distance margin in the feature space and the correlation between the metric and triplet constraints. Next, to enhance robustness to instance perturbation, we propose to enlarge the adversarial margin through minimizing a derived novel loss function termed the perturbation loss. The proposed loss can be viewed as a data-dependent regularizer and easily plugged into any existing metric learning methods. Finally, we show that the enlarged margin is beneficial to the generalization ability by using the theoretical technique of algorithmic robustness. Experimental results on 16 datasets demonstrate the superiority of the proposed method over existing state-of-the-art methods in both discrimination accuracy and robustness against possible noise.