Mining on Manifolds: Metric Learning without Labels

TL;DR

This paper introduces an unsupervised framework for mining hard training examples based on manifold and Euclidean similarity disagreements, enhancing fine-tuning of pre-trained models for classification and retrieval.

Contribution

It proposes a novel unsupervised method for hard example mining using manifold-based similarity measures without labels, improving fine-tuning performance.

Findings

Outperforms prior supervised and unsupervised models in fine-grained classification.

Effective in unsupervised fine-tuning of pre-trained networks.

Applicable to object retrieval tasks.

Abstract

In this work we present a novel unsupervised framework for hard training example mining. The only input to the method is a collection of images relevant to the target application and a meaningful initial representation, provided e.g. by pre-trained CNN. Positive examples are distant points on a single manifold, while negative examples are nearby points on different manifolds. Both types of examples are revealed by disagreements between Euclidean and manifold similarities. The discovered examples can be used in training with any discriminative loss. The method is applied to unsupervised fine-tuning of pre-trained networks for fine-grained classification and particular object retrieval. Our models are on par or are outperforming prior models that are fully or partially supervised.