Generalizable Embeddings with Cross-batch Metric Learning

TL;DR

This paper introduces a novel perspective on global average pooling in deep metric learning by modeling it as a convex combination of learnable prototypes, enabling better generalization to unseen classes through cross-batch regularization.

Contribution

It formulates GAP as a convex combination of prototypes and develops a recursive learning process that improves generalization across batches in deep metric learning.

Findings

Improved generalization to unseen classes.

Effective regularization via cross-batch prototype fitting.

Validated on 4 popular DML benchmarks.

Abstract

Global average pooling (GAP) is a popular component in deep metric learning (DML) for aggregating features. Its effectiveness is often attributed to treating each feature vector as a distinct semantic entity and GAP as a combination of them. Albeit substantiated, such an explanation's algorithmic implications to learn generalizable entities to represent unseen classes, a crucial DML goal, remain unclear. To address this, we formulate GAP as a convex combination of learnable prototypes. We then show that the prototype learning can be expressed as a recursive process fitting a linear predictor to a batch of samples. Building on that perspective, we consider two batches of disjoint classes at each iteration and regularize the learning by expressing the samples of a batch with the prototypes that are fitted to the other batch. We validate our approach on 4 popular DML benchmarks.