Advancing Deep Metric Learning Through Multiple Batch Norms And Multi-Targeted Adversarial Examples

TL;DR

This paper introduces MDProp, a novel framework that enhances deep metric learning by using multiple batch normalization layers and multi-targeted adversarial examples, leading to better performance and robustness across diverse data distributions.

Contribution

MDProp is the first framework to generate multi-targeted adversarial examples in feature space for targeted regularization in deep metric learning.

Findings

Up to 2.95% increase in Recall@1 on clean data.

Up to 2.12 times improved robustness against varied input distributions.

Effective regularization improves generalization of DML models.

Abstract

Deep Metric Learning (DML) is a prominent field in machine learning with extensive practical applications that concentrate on learning visual similarities. It is known that inputs such as Adversarial Examples (AXs), which follow a distribution different from that of clean data, result in false predictions from DML systems. This paper proposes MDProp, a framework to simultaneously improve the performance of DML models on clean data and inputs following multiple distributions. MDProp utilizes multi-distribution data through an AX generation process while leveraging disentangled learning through multiple batch normalization layers during the training of a DML model. MDProp is the first to generate feature space multi-targeted AXs to perform targeted regularization on the training model's denser embedding space regions, resulting in improved embedding space densities contributing to the improved generalization in the trained models. From a comprehensive experimental analysis, we show that MDProp results in up to 2.95% increased clean data Recall@1 scores and up to 2.12 times increased robustness against different input distributions compared to the conventional methods.