Distance-Ratio-Based Formulation for Metric Learning

TL;DR

This paper introduces a distance-ratio-based formulation for metric learning that improves stability and speed, leading to better or comparable classification performance in few-shot learning tasks.

Contribution

It proposes a novel distance-ratio-based formulation for metric learning that is scale-invariant and optimizes classification confidence, enhancing learning stability and efficiency.

Findings

DR formulation enables faster metric learning

It provides more stable training compared to softmax-based methods

Achieves improved or comparable generalization in few-shot classification

Abstract

In metric learning, the goal is to learn an embedding so that data points with the same class are close to each other and data points with different classes are far apart. We propose a distance-ratio-based (DR) formulation for metric learning. Like softmax-based formulation for metric learning, it models $p(y=c|x')$, which is a probability that a query point $x'$ belongs to a class $c$. The DR formulation has two useful properties. First, the corresponding loss is not affected by scale changes of an embedding. Second, it outputs the optimal (maximum or minimum) classification confidence scores on representing points for classes. To demonstrate the effectiveness of our formulation, we conduct few-shot classification experiments using softmax-based and DR formulations on CUB and mini-ImageNet datasets. The results show that DR formulation generally enables faster and more stable metric learning than the softmax-based formulation. As a result, using DR formulation achieves improved or comparable generalization performances.