Context Embedding Networks

TL;DR

This paper introduces Context Embedding Networks (CENs), a novel approach that models worker biases and visual context to learn more interpretable low-dimensional embeddings from crowd-sourced similarity data.

Contribution

CENs jointly learn interpretable embeddings, worker biases, and visual context, overcoming limitations of previous models that assumed uniform similarity criteria or ignored data influence.

Findings

Modeling worker bias improves embedding interpretability.

Incorporating visual context enhances embedding quality.

CENs outperform existing methods on noisy datasets.

Abstract

Low dimensional embeddings that capture the main variations of interest in collections of data are important for many applications. One way to construct these embeddings is to acquire estimates of similarity from the crowd. However, similarity is a multi-dimensional concept that varies from individual to individual. Existing models for learning embeddings from the crowd typically make simplifying assumptions such as all individuals estimate similarity using the same criteria, the list of criteria is known in advance, or that the crowd workers are not influenced by the data that they see. To overcome these limitations we introduce Context Embedding Networks (CENs). In addition to learning interpretable embeddings from images, CENs also model worker biases for different attributes along with the visual context i.e. the visual attributes highlighted by a set of images. Experiments on two noisy crowd annotated datasets show that modeling both worker bias and visual context results in more interpretable embeddings compared to existing approaches.