A preference elicitation interface for collecting dense recommender datasets with rich user information

TL;DR

This paper introduces a novel preference elicitation interface designed to efficiently gather dense, rich user preference data on visual stimuli, aiming to enhance recommender system research and understanding of human preferences.

Contribution

The paper presents a new interface for collecting dense preference datasets with rich user information, bridging recommender systems research with social and behavioral sciences.

Findings

Enables quick and effortless preference data collection.

Facilitates analysis of diversity and psychological effects in preferences.

Supports evaluation of recommender systems through counterfactual experiments.

Abstract

We present an interface that can be leveraged to quickly and effortlessly elicit people's preferences for visual stimuli, such as photographs, visual art and screensavers, along with rich side-information about its users. We plan to employ the new interface to collect dense recommender datasets that will complement existing sparse industry-scale datasets. The new interface and the collected datasets are intended to foster integration of research in recommender systems with research in social and behavioral sciences. For instance, we will use the datasets to assess the diversity of human preferences in different domains of visual experience. Further, using the datasets we will be able to measure crucial psychological effects, such as preference consistency, scale acuity and anchoring biases. Last, we the datasets will facilitate evaluation in counterfactual learning experiments.