Using Shortlists to Support Decision Making and Improve Recommender System Performance

TL;DR

This paper investigates shortlists in recommender systems as a tool to support user decision-making and enhance implicit feedback, leading to better recommendations and user satisfaction.

Contribution

It introduces shortlists as a dual-purpose interface component that aids decision-making and generates valuable implicit feedback for improved recommendation quality.

Findings

Users make better decisions with shortlists.

Users prefer interfaces with shortlists.

Implicit feedback from shortlists nearly doubles recommendation quality.

Abstract

In this paper, we study shortlists as an interface component for recommender systems with the dual goal of supporting the user's decision process, as well as improving implicit feedback elicitation for increased recommendation quality. A shortlist is a temporary list of candidates that the user is currently considering, e.g., a list of a few movies the user is currently considering for viewing. From a cognitive perspective, shortlists serve as digital short-term memory where users can off-load the items under consideration -- thereby decreasing their cognitive load. From a machine learning perspective, adding items to the shortlist generates a new implicit feedback signal as a by-product of exploration and decision making which can improve recommendation quality. Shortlisting therefore provides additional data for training recommendation systems without the increases in cognitive load that requesting explicit feedback would incur. We perform an user study with a movie recommendation setup to compare interfaces that offer shortlist support with those that do not. From the user studies we conclude: (i) users make better decisions with a shortlist; (ii) users prefer an interface with shortlist support; and (iii) the additional implicit feedback from sessions with a shortlist improves the quality of recommendations by nearly a factor of two.