Learning Unified Distance Metric Across Diverse Data Distributions with Parameter-Efficient Transfer Learning

TL;DR

This paper introduces PUMA, a parameter-efficient transfer learning approach for unified metric learning across diverse datasets, outperforming existing models with significantly fewer trainable parameters.

Contribution

The paper proposes PUMA, a novel framework combining a pre-trained model with adapters and prompts to learn a unified metric across heterogeneous data distributions.

Findings

PUMA outperforms state-of-the-art dataset-specific models.

PUMA uses about 69 times fewer trainable parameters.

A new benchmark with 8 datasets was compiled for evaluation.

Abstract

A common practice in metric learning is to train and test an embedding model for each dataset. This dataset-specific approach fails to simulate real-world scenarios that involve multiple heterogeneous distributions of data. In this regard, we explore a new metric learning paradigm, called Unified Metric Learning (UML), which learns a unified distance metric capable of capturing relations across multiple data distributions. UML presents new challenges, such as imbalanced data distribution and bias towards dominant distributions. These issues cause standard metric learning methods to fail in learning a unified metric. To address these challenges, we propose Parameter-efficient Unified Metric leArning (PUMA), which consists of a pre-trained frozen model and two additional modules, stochastic adapter and prompt pool. These modules enable to capture dataset-specific knowledge while avoiding bias towards dominant distributions. Additionally, we compile a new unified metric learning benchmark with a total of 8 different datasets. PUMA outperforms the state-of-the-art dataset-specific models while using about 69 times fewer trainable parameters.