Generalization in Metric Learning: Should the Embedding Layer be the Embedding Layer?

TL;DR

This paper explores the impact of using different layers in deep metric learning models for fine-grained image retrieval, proposing a new regularization approach and demonstrating improved generalization and state-of-the-art results.

Contribution

It introduces a novel regularization method that involves selecting optimal layers for feature extraction in metric learning, enhancing generalization and retrieval performance.

Findings

Improved generalization when using layers other than the embedding layer.

State-of-the-art results on Cars-196, CUB-200-2011, and Stanford Online Product datasets.

Regularization by layer selection benefits fine-grained image retrieval.

Abstract

This work studies deep metric learning under small to medium scale data as we believe that better generalization could be a contributing factor to the improvement of previous fine-grained image retrieval methods; it should be considered when designing future techniques. In particular, we investigate using other layers in a deep metric learning system (besides the embedding layer) for feature extraction and analyze how well they perform on training data and generalize to testing data. From this study, we suggest a new regularization practice where one can add or choose a more optimal layer for feature extraction. State-of-the-art performance is demonstrated on 3 fine-grained image retrieval benchmarks: Cars-196, CUB-200-2011, and Stanford Online Product.