Hierarchical Annotation of Images with Two-Alternative-Forced-Choice Metric Learning

TL;DR

This paper introduces a method that uses two-alternative-forced-choice testing combined with deep metric learning to efficiently uncover hierarchical structures in high-dimensional data like images, reducing annotation effort.

Contribution

It presents a novel approach that leverages discriminative testing and active triplet selection to build hierarchical embeddings without extensive manual annotations.

Findings

Successfully demonstrated hierarchical clustering on synthetic data confirming shape bias.

Extracted finer-grained hierarchical structure on Fashion-MNIST dataset.

Method reduces annotation effort through active test selection.

Abstract

Many tasks such as retrieval and recommendations can significantly benefit from structuring the data, commonly in a hierarchical way. To achieve this through annotations of high dimensional data such as images or natural text can be significantly labor intensive. We propose an approach for uncovering the hierarchical structure of data based on efficient discriminative testing rather than annotations of individual datapoints. Using two-alternative-forced-choice (2AFC) testing and deep metric learning we achieve embedding of the data in semantic space where we are able to successfully hierarchically cluster. We actively select triplets for the 2AFC test such that the modeling process is highly efficient with respect to the number of tests presented to the annotator. We empirically demonstrate the feasibility of the method by confirming the shape bias on synthetic data and extract hierarchical structure on the Fashion-MNIST dataset to a finer granularity than the original labels.