Adaptively Learning the Crowd Kernel

TL;DR

This paper presents an adaptive algorithm that learns a comprehensive similarity matrix from crowdsourced triplet queries, enabling effective embedding of objects into Euclidean space for various domain-specific features.

Contribution

The paper introduces a novel adaptive sampling algorithm that efficiently learns a crowd-based similarity kernel from triplet queries, improving over non-adaptive methods.

Findings

The crowd kernel accurately captures domain-specific features.

Adaptive sampling reduces the number of queries needed.

SVM analysis confirms the kernel's effectiveness.

Abstract

We introduce an algorithm that, given n objects, learns a similarity matrix over all n^2 pairs, from crowdsourced data alone. The algorithm samples responses to adaptively chosen triplet-based relative-similarity queries. Each query has the form "is object 'a' more similar to 'b' or to 'c'?" and is chosen to be maximally informative given the preceding responses. The output is an embedding of the objects into Euclidean space (like MDS); we refer to this as the "crowd kernel." SVMs reveal that the crowd kernel captures prominent and subtle features across a number of domains, such as "is striped" among neckties and "vowel vs. consonant" among letters.