Active Ordinal Querying for Tuplewise Similarity Learning

TL;DR

This paper introduces a novel method called InfoTuple for selecting tuple-based similarity queries to efficiently learn embeddings that reflect human-perceived relative similarities, outperforming triplet-based methods.

Contribution

It generalizes triplet queries to arbitrary-sized tuples and develops an adaptive selection method using mutual information maximization, improving efficiency and consistency.

Findings

InfoTuple outperforms state-of-the-art triplet methods on synthetic and human datasets.

Larger tuples lead to significant gains in query efficiency.

Empirical results demonstrate improved embedding quality with tuple queries.

Abstract

Many machine learning tasks such as clustering, classification, and dataset search benefit from embedding data points in a space where distances reflect notions of relative similarity as perceived by humans. A common way to construct such an embedding is to request triplet similarity queries to an oracle, comparing two objects with respect to a reference. This work generalizes triplet queries to tuple queries of arbitrary size that ask an oracle to rank multiple objects against a reference, and introduces an efficient and robust adaptive selection method called InfoTuple that uses a novel approach to mutual information maximization. We show that the performance of InfoTuple at various tuple sizes exceeds that of the state-of-the-art adaptive triplet selection method on synthetic tests and new human response datasets, and empirically demonstrate the significant gains in efficiency and query consistency achieved by querying larger tuples instead of triplets.